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1.  INTRODUCTION 


When  a  microburst  occurs  near  an  airport,  an  aircraft  on  approach  or  depaiture  may 
experience  hazardous  weather  conditions.  A  system  called  the  Terminal  Doppler  Weather 
Radar  (TDWR)  has  been  developed  to  automatically  detect  the  presence  of  microbursts 
and  other  wind  shear  phenomena  and  to  alert  air  traffic  controllers  of  potential  hazards. 
The  TDWR  scans  the  airport  area  to  obtain  radar  data  both  at  the  surface  and  aloft.  The 
first  phase  of  processing  this  data  involves  the  application  of  several  algorithms  which  iden¬ 
tify  two-dimensional  reflectivity  and  velocity  regions.  The  algorithm  designed  to  detect 
regions  of  rotating  winds  aloft  is  the  subject  of  this  report. 

The  rotation  algorithm  was  first  developed  in  1987,  and  since  then  it  has  been  undergo¬ 
ing  much  testing  and  evaluation.  Though  it  has  been  in  use  operationally  for  TDWR  demon¬ 
strations  and  evaluations  since  1987,  it  has  been  determined  that  further  improvements 
were  necessary  to  reduce  the  incidence  of  spurious  rotation  declarations  and  to  provide 
more  accurate  detection  of  rotation  regions  associated  with  microbursts. 

A  detailed  description  of  the  algorithm  and  its  role  in  the  microburst  recognition  system 
is  given,  and  the  procedures  used  for  improvements  are  discussed.  The  performance  of 
the  algorithm  on  data  from  three  geographical  regions  was  assessed,  and  the  results  were 
analyzed  and  presented  here,  along  with  an  explanation  of  the  technique  of  performance 
evaluation.  The  outcome  of  experimentation  with  simulated  data  is  included  as  well.  Final¬ 
ly,  a  set  of  conclusions  are  drawn,  and  suggestions  for  future  enhancements  are  also  pro¬ 
vided. 


1 


2.  BACKGROUND 


2.1  BRIEF  OVERVIEWS 

2.1.1  Rotation  in  the  Atmosphere 

In  a  vertically  sheared  environment,  the  presence  of  various  instabilities  in  the  atmo¬ 
sphere,  such  as  an  updrafts  and  downdrafts,  can  cause  a  region  of  air  to  rotate.  In  theory, 
as  shown  in  Figure  1 ,  one  way  this  can  happen  is  that  the  vertical  shear  causes  a  rolling 
vortex,  and  when  its  middle  is  forced  upward  by  an  updraft,  the  bisected  cylinder  forms 
two  regions,  now  rotating  in  opposite  directions,  in  which  the  axes  of  rotation  are  perpendic¬ 
ular  to  the  surface.  In  a  storm  cell,  updrafts  are  often  accompanied  by  downdrafts.  The 
downdraft  is  usually  due  to  a  descending  reflectivity  core  (precipitative  loading),  or  water 
phase  changes,  such  as  melting  and  evaporation.  In  response  to  an  accelerating  downdraft, 
the  regions  becomes  elongated,  and  the  radius  of  the  rotating  region  decreases  while  its 
axis  lengthens.  Because  of  the  conservation  of  angular  momentum,  the  region  will  rotate 
even  faster. 

The  aloft  storm  structure  of  a  surface-outflow-producing  microburst  often  contains 
upper-level  divergence,  a  descending  reflectivity  core,  mid-level  convergence,  and  rotation 
in  the  vicinity  of  the  downdraft,  as  shown  in  Figure  2.  In  fact,  78  percent  of  those  micro¬ 
bursts  studied  in  Denver,  1988  had  rotation  regions  associated  with  them.  [1]  Since  these 
characteristics,  or  features,  exist  often  several  minutes  before  the  divergent  outflow  at  the 
surface,  their  presence  can  be  used  to  enhance  detection  and  prediction  of  microbursts. 

2.1.2  The  Microburst  Recognition  Algorithm 

Initially,  the  microburst  algorithm  used  by  TDWR  detected  surface  outflow  using  only 
surface  velocity  data.  Subsequently,  information  was  incorporated  to  deal  with  the  features 
aloft  mentioned  above.  The  current  TDWR  algorithm  employs  a  set  of  algorithms  which 
together  comprise  three  processing  modules:  feature  extraction,  vertical  integration,  and 
microburst  recognition.  [2]  The  feature  extraction  algorithms  identify  regions  of  conver¬ 
gence,  reflectivity  cells,  storm  cells,  reflectivity  cores,  upper-level  divergence,  and  rotation. 
These  regions  are  then  associated  into  three-dimensional  structures  during  the  vertical  inte¬ 
gration  phase.  In  combination  with  the  surface  outflow  information,  and  in  accordance 
with  a  complex  rule  set,  the  structures  lead  to  the  generation  of  microburst  alarms.  These 
features  aloft  aid  the  time/space  correlated  surface  features  in  the  timely  detection  of  low- 
altitude  windshear  by  allowing  an  alert  to  be  issued  while  the  divergent  signature  at  the 
surface  is  still  weak.  [3] 
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Figure  1.  How  Rotation  Can  Occur  In  the  Atmosphere 

Vertical  shear  causes  a  rolling  vortex  [a],  and  when  Its  middle  Is  forced  upward,  the  bisected  cylinder 

forms  two  regions,  rotating  In  opposite  dlrectlons(b] ,  In  which  the  axes 

to  the  surface.  In  a  storm  cell,  updrafts  are  often  accompanied  by  downdrafts.  7^®  ff 

due  to  a  descending  reflectivity  core  (precipitative  loading),  or  water  phase  changes,  su^  “ 

and  evaf^ratlon.  In  response  to  an  accelerating  downdraft,  the  radius  of 

while  Its  axis  lengthens  [c].  Because  of  the  conservation  of  angular  momentum,  the  region  will  rotate 
even  faster. 
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storm  Cell 


Initial  Stage  Descending  Core  Surface  Microburst 


Figure  2.  Storm  Evolution 

Th/s  figure  Illustrates  the  mode!  of  the  use  of  features  aloft  as  microburst  precursors.  As  the  reflective 
core  lengthens  and  descends  within  a  storm  cell,  convergence  and  rotation  may  be  occurring  aloft.  In 
this  model,  the  existence  of  these  features  may  point  to  a  subsequent  divergent  outflow  at  the  surface. 
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2.1.3  The  Scan  Strategy 


The  TDWR  system  detects  microbursts  using  images  collected  from  a  single  Doppler 
radar.  The  beamwidth  of  the  radar  is  0.5  degrees  and  has  a  1  degree  coherent  integration 
interval  angle.  Products  acquired  from  the  radar  include  velocity,  reflectivity,  signal-to- 
noise  ratio,  and  spectrum  width.  The  pulse  repetition  frequency  (PRF)  of  the  radar  is  vari¬ 
able  in  order  to  offset  the  effects  of  range  obscuration,  [4]  though  scans  whose  PRF  are 
below  700  hz  are  not  used  by  the  feature  extraction  algorithms. 

Originally,  only  surface  scans  were  utilized  for  microburst  detection,  but  when  the  im¬ 
portance  of  features  aloft  was  recognized,  it  became  necessary  to  collect  data  at  various 
higher  elevations.  When  the  radar  site  was  in  Huntsville  in  1986,  scans  were  1  degree 
in  elevation  apart,  starting  at  0.0  degrees  and  ending  at  15.0  degrees,  and  the  scans  were 
made  approximately  every  10-13  seconds.  The  scanning  sector  was  about  115  to  120  de¬ 
grees  wide.  In  Denver  and  Kansas  City,  surface  scans  were  done  at  one-minute  time  inter¬ 
vals,  and  between  those  were  five  or  six  scans  aloft  at  roughly  equally  spaced  elevations, 
up  to  40  degrees,  so  as  to  provide  fairly  dense  coverage  (no  gaps  greater  than  1  km  in 
height)  up  to  an  altitude  of  at  least  6  km.  This  strategy  allows  for  the  same  elevation  to 
be  scanned  about  every  2,5  minutes.  [5] 
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3.  DESCRIPTION  OF  THE  ROTATION  ALGORITHM 


3.1  GENERAL  DESCRIPTION  OF  BASE  VERSION  1.4 

The  algorithm  is  based  loosely  on  a  pattern  recognition  algorithm  developed  for  azi¬ 
muthal  shear  in  mesocyclones.  [6]  The  rotation  algorithm  identifies,  in  real  time,  regions 
of  cyclonic  and  anti-cyclonic  rotation  by  first  searching  radar  image  data  for  runs  of  azi¬ 
muthal  shear.  These  runs,  or  shear  segments,  are  then  associated  together  by  range  and 
by  azimuth  into  regions.  The  output  of  the  algorithm  is  a  set  of  bounding  boxes,  which 
are  axis-aligned  and  enclose  the  x-y  (cartesian  coordinate)  extents  of  the  region.  A  flow 
diagram  illustrating  this  process  is  shown  in  Figure  3. 

The  use  of  a  single  Doppler  radar  allows  for  only  the  radial  component  of  wind  veloci¬ 
ties  to  be  sensed,  and  therefore  the  identification  of  azimuthal  shear  can  only  imply  a  rotat¬ 
ing  wind  field.  This  situation  is  illustrated  in  Figure  4a.  Here,  the  solid  curves  are  isodops 
representing  velocities  moving  away  from  the  radar,  whereas  the  dashed  lines  indicate  velo¬ 
cities  moving  toward  the  radar.  The  region  depicted  is  an  idealization  of  a  large-scale 
vortex,  [7]  and  results  in  the  signature  in  Figure  4b.  An  actual  radar  image  is  given  in 
Figure  5,  in  which  rotation  regions  are  indicated  by  red  circles. 

3.2  DETAILED  DESCRIPTION  OF  BASE  VERSION  1.4 

3.2.1  Preliminary  Processing 

Input  to  the  algorithm  is  polar  radar  data,  which  has  been  pre-processed  with  velocity 
de-aliasing  and  signal-to-noise  thresholding,  using  a  set  of  adjustable  parameters.  The 
data,  in  the  form  of  radial  velocity  vectors,  is  read  in  by  the  algorithm.  The  first  radial 
is  buffered,  and  various  preliminary  calculations  are  performed.  For  example,  these  in¬ 
clude  the  specification  of  the  processing  interval.  This  is  the  calculation  of  the  range  over 
which  the  algorithm  will  process  data,  taking  the  elevation  of  the  scan  into  account,  and 
given  that  we  are  only  concerned  with  data  lying  between  1  and  6  km  AGL  in  altitude  and 
no  further  than  35  km  in  range  from  the  radar. 

3.2.2  Starting  Segments 

As  soon  as  two  radials  have  been  read  in,  the  search  for  both  cyclonic  and  anti-cyclonic 
azimuthal  shear  begins.  The  velocity  values  within  the  two  radials  are  compared  at  corre¬ 
sponding  range  gates.  If  the  absolute  value  of  the  difference  between  the  velocities  in  the 
two  radials  is  greater  than  veMhresh,  a  threshold  currently  set  to  0,  then  a  shear  segment 
is  started.  In  other  words,  the  starting  point  of  a  shear  segment  is  established  when  a 
velocity  value  is  either  monotonically  increasing  or  decreasing  with  respect  to  the  velocity 
azimuthally  adjacent  to  it  in  the  scan.  The  direction  of  the  velocity  change  is  called  the 
tendency. 


7 


Figure  3.  Algorithmic  Process  Flow 

The  rotation  algorithm  Identifies,  In  real  time,  regions  of  cyclonic  and  antl-cyclonlc  rotation  by  first  search¬ 
ing  radar  Image  data  for  runs  of  azimuthal  shear.  These  runs,  or  shear  segments,  are  then  associated 
spatially  by  range  and  azimuth  Into  regions.  The  output  of  the  algorithm  Is  a  set  of  bounding  boxes, 
which  are  axis-aligned  and  enclose  the  x-y  (cartesian  coordinate)  extents  of  the  region. 
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Figure  4.  Rotation  as  Observed  by  Single  Doppler 

The  use  of  a  single  Doppler  radar  allows  for  only  the  radial  component  of  wind  velocities  to  be  sensed, 
and  therefore  azimuthal  shear  can  only  Imply  a  rotating  wind  field.  This  situation  Is  illustrated  in  Figure 
4a.  Here,  the  solid  curves  are  Isodops  for  representing  velocities  moving  away  from  the  radar,  whereas 
the  dashed  lines  are  the  velocities  moving  toward  the  radar.  This  Is  an  Idealization  of  a  large-scale  vortex, 
{7]  and  results  In  a  signature  such  as  that  In  depicted  In  Figure  4b. 
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3.2.3  Continuing  Segments 

To  process  the  next  radial  of  radar  data,  its  velocities  are  now  compared  to  those  in 
the  previous  radial.  For  those  range  gates  where  shear  segments  have  already  been  started, 
the  decision  now  is  whether  to  add  another  point  to  the  segment.  If  the  velocity  at  this 
point  continues  to  change  in  the  same  direction  as  it  had  been  (that  is,  the  tendency  remains 
the  same),  the  segment  is  allowed  to  continue. 

3.2.4  Ending  Segments 

While  building  a  shear  segment,  if  the  tendency  changes,  then  the  shear  segment  is 
ended.  The  completed  shear  segment  is  now  subjected  to  a  number  of  tests,  which  will 
further  determine  if  it  should  be  retained. 

3.2.5  Validating  Segments 

Each  shear  segment  must  have  a  velocity  difference  between  its  endpoints  of  5  m/s 
for  cyclonic  rotation  (or  -5  m/s  for  anti-cyclonic  rotation).  It  must  have  the  minimum 
length  restricted  by  min_seg_len,  currently  set  to  1  km,  but  the  shear  segment  can  be  no 
more  than  5  km  long.  If  it  also  spans  3  degrees  in  azimuth,  it  is  retained,  and  the  shear 
segment  will  pass  to  the  next  phase  of  processing. 

3.2.6  Merging  into  Regions 

All  of  the  segments  found  on  the  scan  are  collected  and  separated  in  cyclonic  and  anti- 
cyclonic  groups.  Each  of  the  segments  in  these  groups  are  compared  to  all  the  others  in 
that  group  to  see  if  they  correlate  spatially.  In  order  to  do  this,  the  two  segments  being 
compared  may  be  no  more  than  1000  meters  apart  in  range,  and  they  must  overlap  in  azi¬ 
muth.  These  two  thresholds  are  called  range_thr  and  az_thr,  respectively.  If  the  two  seg¬ 
ments  satisfy  these  criteria,  then  they  are  merged,  or  considered  to  be  part  of  the  same 
region.  The  segments  themselves  remain  as  separate  entities,  but  now  together  they  denote 
a  region. 

3.2.7  Output  of  Valid  Regions 

When  all  possible  regions  have  been  formed  from  the  shear  segments,  attributes  such 
as  centroid  location  and  average  shear  are  calculated.  Further  testing  against  thresholds 
is  performed:  the  maximum  velocity  difference  in  the  region  must  be  greater  than  or  equal 
to  the  deltav  threshold,  currently  10  m/s,  its  centroid  must  lie  no  farther  than 
35  km  in  range,  £md  it  must  have  a  minimum  area  of  1  sq-km,  corresponding  to  area_thr, 
while  being  no  more  than  15  sq-km  in  area.  The  velocity  threshold  was  chosen  to  be 
10  m/s  [8]  because  it  was  the  threshold  used  in  a  previous  analysis  [9]  and  because  a  similar 
radial  velocity  associated  with  a  microburst  is  considered  also  to  be  hazardous.  Those  re¬ 
gions  satisfying  these  criteria  become  the  final  output  of  the  algorithm. 
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Figure  5.  TDWR  Image,  Showing  Rotation  Region 

This  scan  Is  from  the  radar  site  at  Kansas  City,  on  July  1,  1989  at  21:54:04  UT.  The  elevation  of  the 
scan  Is  11  degrees,  range  rings  ere  10  km  apart,  and  azimuth  lines  are  10  degrees  apart.  Colors  are 
denoted  as  In  the  legend.  Indicating  mfs.  The  red  circles  are  rotation  regions  Identified  by  a  human  expert. 
These  two  regions  were  the  only  ones  on  this  scan  to  meet  the  various  truthing  criteria  of  velocity  differen¬ 
tial,  area,  range,  and  association  with  a  mIcroburst-producIng  storm  cell.  Any  other  azimuthal  shear 
that  may  be  observed  here  Is  considered  spurious  Insofar  as  microburst  detection  Is  concerned. 


4.  IMPROVEMENT  AND  EVALUATION  PROCEDURE 


4.1  OBJECTIVES 

The  approach  taken  involved  a  preliminary  evaluation,  which  led  to  improvements  and 
testing,  and  then  a  final  evaluation.  The  motivations  for  making  improvements  to  the  algo¬ 
rithm  were  numerous.  First,  according  to  the  preliminary  evaluation,  it  appeared  most 
important  to  decrease  the  very  high  probability  of  false  detections  (PFD).  At  the  same 
time,  it  was  desirable  to  increase  the  probability  of  detection  (POD).  It  was  of  somewhat 
secondary  importance  to  decrease  the  ratio  of  the  total  number  of  detections  (whether  ulti¬ 
mately  correct  or  incorrect)  to  the  total  number  of  truth  observations  (the  DTR,  or  the  detec- 
tions-to-truth-ratio).  These  are  the  basic  aims,  though  the  particular  values  required  for 
the  POD  and  PFD  are  unspecified  at  present. 

It  is  typically  the  issue  that  a  valid  microburst  rotation  region  will  give  rise  to  two  regions 
of  azimuthal  shear,  as  illustrated  in  Figure  4b.  However,  it  does  complicate  overall  TDWR 
region  association  if  more  than  two  azimuthal  shear  regions  are  detected  for  a  given  truth 
region.  This  makes  it  desirable  to  reduce  the  number  of  detected  regions  in  relation  to 
the  number  of  truth  observations,  the  DTR. 

The  task  of  improving  an  algorithm  can  involve  two  types  of  changes:  logic  changes 
and  parameter  tuning.  The  logic  changes  represent  the  infusion  of  new  ideas  into  the  algo¬ 
rithm  processing,  involving  changes  in  the  intrinsic  design,  whereas  the  parameter  tuning 
implies  site  adaptation  parameter  changes  that  can  be  made  without  modifying  the  TDWR 
basic  software.  Both  of  these  techniques  were  employed  in  improving  the  performance 
of  the  rotation  algorithm. 

4.2  OVERALL  TESTING  STRATEGY 

The  approach  taken  for  this  study  was  to  make  logic  changes  and  parameter  tuning 
on  two  types  of  data  sets,  one  small  and  one  large.  First,  a  small  set  of  data  would  be 
gathered  and  tests  would  be  performed  on  it  to  aid  in  determining  the  characteristics  of 
the  final  version  of  the  algorithm.  Then,  the  revised  algorithm  would  be  run  on  the  large 
data  set  in  order  to  obtain  statistically  significant  performance  scoring.  In  this  report,  the 
small  set  will  be  referred  to  as  “test  data,”  and  the  large  set  will  be  called  “case  data.” 

4.3  DESCRIPTION  OF  DATA  SETS 

4.3.1  Test  Data 

A  test  data  set  was  used  mainly  for  practical  reasons,  given  the  length  of  time  which 
would  be  needed  to  run  the  algorithm  and  the  number  of  permutations  of  the  parameter 
set.  However,  there  is  a  difficulty  inherent  in  the  task  of  finding  representative  data,  then 
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drawing  global  conclusions  based  on  it.  There  is  no  such  thing  as  a  perfect  weather  model 
for  a  given  geographical  region.  The  methodology  used  to  obtain  a  test  data  set  was  as 
follows:  select  case  dates  containing  microburst  events  from  three  geographical  regions 
where  the  radar  has  been  situated.  These  events  should  be  typical  for  their  geographic 
regions.  Then  select  individual  scans  from  these  events  where  the  baseline  algorithm’s 
performance  category  was; 

•  miss  —  a  failure  to  detect 

•  hit  —  a  successful  detection 

•  false  —  a  spurious  detection 

These  performance  categories  will  be  explained  in  more  detail  in  Chapter  5.  Table 
1  below  relates  to  Figure  6  as  follows:  for  each  group  of  scans  in  the  table,  the  site  from 
which  data  was  taken  is  shown  along  with  the  issue  the  scans  were  chosen  to  represent, 
i.e.,  why  those  particular  scans  were  selected  for  test  purposes. 


Table  1. 

Scan  Information 


SCANS 

srre 

ISSUE 

1-6 

Kansas  City 

False 

7-9 

Denver 

Miss 

10-12 

Huntsville 

Hit 

13-20 

Kansas  City 

Miss 

Tape  Translated  on  06/01/90 

Proiect  FLOWS 


c>acMC4cac<3u>sorter^c>9m^cMoac4cgcM€MU>c>9 


+  I  +  +  + 


•fill 


+  I  +  I  +  I  + 


inu)mi/>m«-*vo^csooocMU>r«.€oo^«^os 

♦H  <SJ  ^  ^  ^m4  ^ 


i/>i/>i/>mi/)u>r^<or^osortei/>4r>u>intAi/>t/>i/> 

r^car^r^r^cao\ooscvaoo^c4rwC4rwoaDw^o& 

CM  CM  CM  ^  (/><0  CO  <0  l/>  t/>  CM  CM  CM  i/>  ^ 

^  ^  ^ooco  ^  ^ 


o  o  o  o  o  o 


o  o  o  o  o  o  o 


mmmmi/>mooocoooi/>i/>mtnioi/>i/>i/> 

CMCO<OCMCOOSOOOOSOOf^CMI/>CMI/>^C^^ 
CM  CM  ^  CO  ^  CO  ^0  ^  ^  CM  CM  CM  ^  CM 

^  ^  CM  CM  CM  ^  ^  ^  ^  ^ 


ou^<o^o^ocoosc^^(no^^cot/>CMO^M>^ln 

o<oooo<oo»^cMCM*^'*^^^moo^^Tr 

mcs.oo^oco^o^oM><oo^<oco<o‘^^^coo 

mino^o«-<oooooou>w>i/>u>i/>iocM<o 

^o^o^^^^oso\<o<o<ooscMCM^^^»i•^’^osG^ 

OOOOOOCMCMCM^^CMCMCMCMCMCMCMCMOO 


OOOOOOOOOOO 


o  o  o  o  o  o 


'ooco^-^^coocoinc^^^coooN.^^oso 

m<omoocMO^^^mo<o^i/>i/>o^cM’^ 

^C>-0>O'0C0M9M>^<00S'^<0<0<0<0^’^C0O 

mi/>in^o^oooou>omu7u7mi/>mcMro 

SOSOVOC^OSOCOCOCOOS^CM^^^^^^^OS 

OOOOOOCMCMCM^CMCMCMCMCMCMCMCMOO 


cooor»-<or«-cocococoooor^rwrs.ooor^o 

^O'^^OVOCMOOOOOOOC^0\O^CMCMCMOCM 

o^osovo^r^r^r^r^c^oscMO^r^r^r^CMCMCMr^cM 


<0  m 

o  o 
<o 

^  « 

«  E 

i2  5 

"o  -5 
^  •) 


•-•cMCO'^m^r^coaso 


^CM<o^4n%or^cooso 


<6  o 

s  « 

5  Q) 

•9*  « 

i  p: 


15 


degrees)  covered  In  the  scanning  sector,  the  elevation  In  degrees,  the  direction  of  the  scan  (■¥;  clockwise,  -:  counter-clockwise),  and  the  number 
of  radlals  In  the  scan. 


4.3.2  Case  Data 


The  cases  chosen  for  this  data  base  represent  a  diversity  of  meteorological  conditions. 
Weather  in  Kansas  City,  MO,  July  30,  1989  was  characterized  by  large-scale  hail-  generat¬ 
ing  thunderstorms  with  high  reflectivities.  The  cells  which  developed  in  Huntsville,  AL 
on  July  25,  1986  and  in  Kansas  City,  MO  on  July  1st,  1989  were  isolated  air  mass  thunder¬ 
showers  with  moderate  to  high  reflectivities,  while  the  microbursts  in  Denver,  CO  on  Sep¬ 
tember  2,  1987  were  classified  as  dry  due  to  the  lower  reflectivities.  [8] 

In  this  data  set,  there  were  26  microburst  events  which  were  determined  by  an  experi¬ 
enced  radar  meteorologist  to  have  rotation  precursors. 

4.4  CHANGING  THE  ALGORITHM 

4.4.1  Logic  Improvements 

A  number  of  image  processing  techniques  were  tried,  but  in  most  cases  performance 
was  not  improved,  and  in  some  cases  performance  was  worse.  Among  the  techniques  used 
was  a  three-radial  buffer  as  an  implementation  of  a  sliding  window  to  calculate  average 
velocity  points,  which  was  an  attempt  at  noise  filtering.  This  technique  differed  from  that 
described  in  sections  3.2.2  and  3.2.3  in  that  the  velocities  being  compared  were  not  those 
belonging  to  successive  radials.  Instead,  the  tendency  computation  described  section  3.2.2 
was  performed  with  radials  1  and  3,  radials  2  and  4,  and  so  on,  in  order  to  help  decrease 
the  algorithm’s  susceptibility  to  noise  and  spurious  events.  The  technique  was  implemented 
as  a  substitution  of  the  original  tendency  computation,  with  the  existing  thresholds  being 
applied  subsequently.  The  technique  was  not  being  used  as  a  true  filter,  whereby  it  would 
pre-process  the  raw  data  before  any  velocity  calculations  were  performed.  Since  the  noise 
inherent  in  the  data  was  not  uniform,  the  use  of  this  technique,  in  combination  with  the 
existing  logic  of  the  algorithm,  caused  a  major  drop  in  detection  capability.  In  retrospect, 
a  more  effective  way  to  use  this  technique  would  have  been  to  apply  a  smoothing  filter 
first. 

The  logic  change  that  proved  successful,  however,  was  the  incorporation  of  the  shear 
validation  test  into  the  base  version,  and  this  produced  Version  1.5.  The  purpose  of  the 
test  was  to  keep  segments  from  growing  too  long,  which  would  happen  frequently.  If  seg¬ 
ments  were  allowed  to  include  regions  of  very  weak  shear,  the  total  area  of  a  potential 
rotation  region  would  exceed  the  maximum  area  threshold,  causing  it  to  be  eliminated  from 
the  final  output.  Furthermore,  segments  that  had  become  too  long  would  be  invalidated 
on  the  basis  of  their  length,  which  could  also  cause  the  algorithm  to  miss  a  detection. 

Though  the  shear  validation  test  could  be  used  for  decreasing  the  number  of  algorithm 
misses,  it  could  also  help  eliminate  a  number  of  false  detections.  By  restricting  the  segments 
that  were  retained  to  have  a  higher  overall  shear  value,  the  resulting  regions  detected  are 
more  likely  to  be  physically  meaningful.  A  higher  azimuthal  shear  often  indicates  a  stronger 
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downdraft,  which  in  turn  can  lead  to  a  more  intense  surface  outflow.  Therefore,  this  would 
help  reduce  the  incidence  of  false  detections. 

The  shear  validation  test  works  as  follows:  Assume  that  the  first  two  radials  of  data 
have  been  received.  Segments  are  started  as  in  Version  1.4  (see  section  3.2.2),  but  now 
the  shear  between  the  two  points  of  the  segment  is  calculated  and  saved  for  later  compari¬ 
sons.  When  the  next  radial  of  radar  data  is  processed,  its  velocities  are  now  compared 
to  those  in  the  previous  radial,  as  before.  For  those  range  gates  where  segments  have 
already  been  started,  and  if  the  tendency  computation  indicates  it,  the  decision  now  is  wheth¬ 
er  to  add  another  point  to  the  segment.  In  order  to  make  this  determination,  the  shear 
between  the  candidate  point  and  the  segment  starting  point  is  calculated  and  compared 
to  the  saved  shear  value.  If,  by  adding  this  point  to  the  segment  the  shear  has  not  dropped 
more  than  x%,  where  x  is  shear_drop,  a  threshold  cr.rrently  set  to  80,  then  the  segment 
is  allowed  to  continue,  while  the  saved  shear  shear  value  is  updated. 

This  concept  of  this  test  is  represented  graphically  in  Figure  7.  The  shear  segment 
is  started  at  point  a  and  the  shear  is  recorded  at  point  b,  as  shown  connected  by  the  solid 
line.  Though  the  velocity  differential  between  azimuths  drops  with  point  c,  the  shear  mea¬ 
sured  at  that  point  passes  the  threshold  and  the  current  segment  is  now  shown  as  a  dashed 
line.  The  segment  continues  to  d,  as  shown  by  the  bold  line.  However,  at  point  e,  even 
though  the  velocity  is  still  increasing,  the  shear  drops  too  low,  and  the  segment  is  ended. 
Without  the  use  of  this  technique,  the  segment  would  have  continued  on  through  point  f 
as  indicated  by  the  dotted  lines. 

The  effectiveness  of  this  technique  was  evaluated  by  designing  into  it  a  threshold  that 
could  be  externally  varied  along  with  the  other  parameters  for  the  algorithm.  This  setting 
of  this  threshold,  shear_drop,  allowed  the  shear  validation  test  to  range  from  being  com¬ 
pletely  disabled  to  being  very  restrictive,  and  the  usefulness  of  the  test  could  be  determined 
quantitatively. 

4.4.2  Parameter  Optimization 

Optimization  was  accomplished  by  varying  five  parameters; 

a.  shear^drop  —  The  fraction  by  which  the  current  shear  of  a  segment 
is  allowed  to  drop  and  still  continue  the  segment  growth. 

b.  range__tlir  —  A  variable  whose  value  determines  the  maximum  dis¬ 
tance  between  two  candidate  shear  segments  in  order  that  they  be  con¬ 
sidered  as  part  of  the  same  region. 
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velocity  (m/s) 


Figure  7.  Shear  Validation  Test 

Each  •  represents  a  velocity  date  point.  The  shear  segment  Is  started  at  point  a  and  the  shear  Is  recorded 
at  point  b,  as  shown  connected  by  the  solid  line.  Though  the  velocity  differential  between  azimuths  drops 
with  point  c,  the  shear  measured  at  that  point  passes  the  threshold  end  the  current  segment  Is  now  shown 
as  a  dashed  line.  The  segment  continues  to  d,  as  shown  by  the  bold  line.  However,  at  point  e,  even 
though  the  velocity  Is  still  Increasing,  the  shear  drops  too  low,  and  the  segment  Is  ended.  Without  the 
use  of  this  technique,  the  segment  would  have  continued  on  through  point  f  as  Indicated  by  the  dotted 
lines.  The  velocity  values  ere  representational  and  do  not  conform  fo  the  algorithm  thresholds. 
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c.  area_thr  —  The  minimum  area  required  for  a  valid  output  region. 

d.  az_thr  — The  amount  by  which  the  two  angular  arcs  represented  by 
segments  must  overlap.  A  negative  number  for  this  parameter  implies 
the  size  of  a  gap  instead  of  overlap. 

e.  mm_seg_len  —  The  minimum  length  of  a  valid  shear  segment. 

These  parameters  fall  into  three  categories:  segment  growing,  segment  association, 
and  region  validation.  These  represent  the  main  functions  of  the  algorithm  and  can  have 
a  potentially  significant  effect  on  its  performance.  It  was  reasonable  that  if  the  criteria 
for  segment  growing  was  made  more  restrictive,  then  the  association  step  should  be  more 
lenient  in  order  to  offset  the  possibility  of  missing  detections.  It  was  not  so  clear,  however, 
what  would  be  the  effect  of  changing  area_thr,  since  its  value  is  based  on  the  meteorological 
characteristics  of  actual  rotation  regions. 

The  values  chosen  for  the  parameters  tended  to  be  somewhat  empirical  in  nature, 
though  there  was  an  overall  scheme  to  which  the  testing  adhered:  variables  were  to  be 
changed,  when  possible,  only  one  at  a  time  in  order  to  insure  independent  evaluation  of 
the  performance.  After  each  change,  the  current  POD  and  PFD  were  calculated.  As  long 
as  the  POD  continued  to  increase  and  the  PFD  continued  to  decrease  as  a  result  of  this 
change,  the  variable  was  increased  in  the  same  direction  until  the  performance  worsened. 
At  that  point,  the  variable  was  regressed  to  the  last  value  that  produced  a  positive  result. 

This  procedure  was  continued  for  each  of  the  five  parameters  until  results  degraded 
or  testing  became  impractical.  The  parameter  settings  for  thirteen  of  the  most  significant 
test  runs  are  shown  in  Table  2.  The  last  set  of  satisfactory  results  determined  the  version 
to  be  run  using  the  case  data  to  produce  the  final  performance  scores. 

Table  2. 

Parameter  List:  Values  of  the  Parameters  Used  for  Thirteen  Distinct  Tests  of  the  Algorithm 


Values  for  each  test 

Parameter 

1 

2 

3 

4 

5 

6 

mm 

8 

9 

10 

11 

12 

13 

shear_drop* 

KM 

id 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

range_thr  (m) 

1000 

1000 

1000 

1000 

500 

500 

500 

500 

500 

500 

240 

240 

240 

areajhr  (sq- 
km) 

1 

1 

1 

1 

1 

2 

1.5 

1.5 

2 

2 

1 

1.5 

1 

azjhr  (deg) 

KM 

■■ 

a 

mm 

D 

0 

mm 

-1.3 

-1.3 

-2.3 

-2.3 

-2.3 

-2.3 

min^segjen 

(km) 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

H 

*a  multiplier  having  no  units 
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5.  SCORING  PROCEDURE 


Scoring  is  the  procedure  of  systematically  comparing  algorithm  output,  called  bounding 
boxes,  to  that  of  a  human  expert  operating  with  the  same  set  of  data.  The  observations 
of  the  expert  are  called  ground  truth,  or  simply,  “truth,”  and  the  algorithm  was  scored  for 
only  those  rotation  events  within  a  case  date  where  truth  was  available.  In  order  to  make 
the  comparison  of  bounding  boxes  and  truth,  it  is  important  that  there  be  a  consistent  set 
of  rules  governing  the  method  of  declaring  hits,  misses  and  false  detections.  In  the  following 
discussions,  the  terms  detection  and  region  will  be  used  interchangeably,  according  to  con¬ 
text,  in  order  to  refer  to  algorithm  output. 


5.1  THE  SCORING  PROGRAM 

Consistency  in  scoring  is  maintained  through  the  use  of  an  automatic  scoring  program 
called  ASP.  As  its  input,  this  program  uses  a  database  containing  the  algorithm  bounding 
boxes  and  a  set  of  truth  generated  for  the  same  data.  The  program  compares  all  the  de¬ 
tected  regions  on  a  scan  with  all  the  truth  on  the  same  scan  and  produces  a  set  of  statistics 
which  quantify  the  algorithm’s  ability  to  detect  rotation  regions  relative  to  the  criteria  used. 


5.1.1  Temporal/Spatial  Criteria 

If  a  region  is  within  2  km  in  distance  from  a  truth  observation  on  the  same  scan,  ASP 
declares  this  a  successful  detection,  called  a  hit,  and  the  region  and  truth  are  said  to  overlap. 
An  illustration  of  this  relationship  is  shown  in  Figure  8.  The  scoring  program  also  compares 
the  regions  on  a  scan  with  the  truth  present  on  previous  and  future  scans.  This  type  of 
comparison  provides  information  on  timeliness  and  identifies  a  detection  that  is  considered 
to  be  early  or  late.  If  a  region  existing  on  the  current  scan  overlaps  truth  that  occurs  no 
more  than  two  scans  or  two  minutes  later  (whichever  is  later),  then  this  detection  is  consid¬ 
ered  early.  Likewise,  if  the  truth  overlapping  this  region  instead  occurred  two  scans  or 
two  minutes  earlier,  then  this  is  considered  to  be  a  late  detection.  Note  that  this  is  a  two-di¬ 
mensional  comparison  and  neglects  the  elevations  of  successive  scans,  which  are  deter¬ 
mined  by  the  particular  scanning  strategy  being  used  (see  Section  2.1.3).  If  no  correlation 
with  truth  can  be  found  within  the  above  criteria,  then  the  detection  is  declared  false. 


The  results  of  these  comparisons  are  given  from  two  points  of  view:  regions  that  hit/do 
not  hit  truth,  and  truth  that  hits/does  not  hit  regions,  which  therefore  indicates  false  detec¬ 
tions,  hits  and  misses.  The  rotation  algorithm  can  conceivably  generate  many  regions  within 
the  vicinity  of  a  single  truth  observation.  In  this  case,  the  truth  is  considered  as  being  hit 
once,  and  each  of  the  regions  is  regarded  as  hitting  truth.  Therefore,  the  number  of  region 
hits  may  be  greater  than  the  number  of  truth  hits.  The  case  of  multiple  and  single  regions 
hitting  truth  are  illustrated  in  Figure  8. 
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Figure  8.  Algorithm/Truth  Scoring  Plot 

ASP  declares  a  hit  where  the  region  and  truth  overlap.  Circles  represent  truth,  rectangles  are  algorithm 
output.  Tick  marks  are  5  km  apart.  Largest  circle  Is  at  30  km  from  radar,  located  at  the  center.  This 
plot  Is  of  data  from  July  25,  1986,  Huntsville,  AL,  at  22:01:01  GMT.  The  scan  was  at  an  elevation  of 
9  degrees,  with  a  PRF  of  900. 
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5.1.2  POD,  PFD,  and  DTR 


When  all  comparisons  for  the  case  have  been  made,  various  performance  statistics  are 
computed.  The  total  number  of  truth  hits  divided  by  the  total  number  of  truth  observations 
gives  the  probability  of  detection  (POD)  for  the  algorithm.  The  PFD,  or  probability  of  a 
false  detection,  is  calculated  as 

1  -  (total  number  of  region  hits)  /  total  number  of  regions 

The  number  of  region  hits  is  determined  by  taking  the  total  number  of  regions  gener¬ 
ated,  then  subtracting  the  number  of  false,  early,  or  late  detections.  In  this  calculation, 
therefore,  early  and  late  detections  are  not  being  considered  as  hits. 

The  detections-to-truth-ratio  (DTR)  is  calculated  by  dividing  the  total  number  of  de¬ 
tected  regions  by  the  total  number  of  truths.  This  ratio  gives  a  measure  of  the  algorithm’s 
“neatness,”  indicating  the  degree  to  which  it  is  being  discriminating  in  its  issuance  of  detec¬ 
tion.  Ideally,  it  is  desirable  to  have  a  DTR  as  close  to  unity  as  possible. 

5.2  EVENT  SCORING 

5.2.1  DeHnitions 

The  scoring  program  also  provides  another  aspect  of  evaluating  algorithm  performance, 
which  is  the  computation  of  an  event  recognition  rate  (RR).  By  definition,  an  event  is  a 
group  of  truth  observations  occurring  over  time  within  the  life  span  of  a  microburst.  When 
truth  is  generated  by  the  meteorologist,  the  rotation  observation  identified  is  associated 
with  a  ntimed  microburst  event.  A  separate  association  post-algorithm-processing  opera¬ 
tion  is  performed  on  the  algorithm  output  so  that  it  can  be  scored  on  an  event  basis  because 
the  rotation  algorithm  itself  does  not  group  its  regions  into  events. 

The  association  program  groups  regions  into  events  by  evaluating  temporal,  spatial, 
directional,  and  altitude  nearness  criteria.  Once  the  regions  have  been  put  into  events, 
the  RR  is  determined  as  follows.  If  at  least  one  region  hits  a  truth  at  any  time  during  the 
truth  event,  that  event  is  considered  as  having  been  recognized  by  the  algorithm.  If  a  detec¬ 
tion  occurs  early  with  respect  to  the  first  observation  of  an  event,  then  the  event  has  been 
recognized  early.  Similarly,  if  a  detection  occurs  late  with  respect  to  the  last  observation 
of  the  event,  then  the  event  has  been  recognized  late.  The  RR  is  the  number  of  recognized 
events  divided  by  the  total  number  of  events. 

Event  scoring  is  interesting  because  the  POD  is  usually  much  higher  than  with  scoring 
on  an  observation-by-observation  basis.  It  shows  that  the  algorithm  is  able  to  detect  some 
part  of  an  event,  and  probably  missed  hitting  some  observations  of  the  event  along  the 
way  because  of  inconsistencies  in  the  data.  The  existence  of  early  and  late  recognition 
of  the  event  may  be  significant  with  regard  to  microburst  prediction  capabilities,  since  the 
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more  timely  the  detections  for  precursors  such  as  rotation,  the  higher  the  confidence  level 
of  the  initial  outflow  detection.  [9]  Late  recognitions  of  events  indicates  over-sensitivity 
of  the  algorithm,  causing  it  to  continue  to  identify  rotation  regions  even  after  the  windfield 
has  started  to  decay,  implying  the  need  for  further  parameter  adjustment. 

5.3  TRUTH  GENERATION 

5.3.1  Truthing  Conditions 

When  scoring  the  performance  of  an  algorithm  it  is  important  to  take  into  consideration 
the  conditions  under  which  the  truth  was  generated.  As  stated  in  section  3.2.1,  the  algorithm 
operates  on  pre-processed  data  in  order  to  reduce  the  amount  of  data  ambiguities.  The 
meteorologist,  however,  works  with  raw  data  and  performs  visual  interpolations  over  the 
incongruities  of  the  data.  This  is  especially  significant  with  regard  to  signal-to-noise  thresh¬ 
olding,  and  the  ramifications  of  this  will  be  discussed  in  Chapter  6. 

When  a  rotation  region  is  identified  by  the  truther,  the  truth  observation  is  expressed 
as  a  line  segment  spanning  the  two  points  that  have  the  maximum  velocity  difference  within 
a  rotation  region.  For  scoring  purposes,  the  line  segment  is  considered  to  be  the  diameter 
of  a  circle.  This  circle  is  used  for  the  geographical  comparisons  with  the  algorithm  regions. 
As  a  result,  this  circle  may  have  an  inaccurate  area  compared  to  the  original  rotation  region 
identified  by  the  expert. 

In  general,  the  meteorologist  confines  the  search  for  rotation  regions  to  the  vicinity 
of  microbursts  and  storms  by  identifying  cells  according  to  reflectivity  characteristics.  Con¬ 
ditions  which  give  rise  to  other  types  of  rotation  are  ignored  by  the  truther.  The  rotation 
algorithm,  on  the  other  hand,  has  no  knowledge  of  existing  reflectivity  regions  and  is  there¬ 
fore  not  as  discriminating.  More  importantly  though,  the  reflectivity  information  is  used 
by  the  microburst  recognition  system  after  the  feature  extraction,  so  it  is  not  clear  that 
providing  reflectivity  data  during  feature  extraction  would  improve  overall  performance  of 
the  microburst  system. 

The  expert  can  also  perform  a  time  correlation  with  the  data,  looking  forward  and  back¬ 
ward  over  the  life  span  of  the  event  in  order  to  make  decisions  about  the  existence  of  rota¬ 
tion  regions.  This  sort  of  tiine  history  association  is  not  a  part  of  the  baseline  microburst 
algorithm. 

5.3.2  General  Truth  Characteristics 

Figures  9, 10,  and  11  are  the  results  of  a  study  to  characterize  rotation  truth  generated 
for  Huntsville  and  Denver  data.  The  histogram  in  Figure  9  shows  that  most  of  the  line 
segments  found  by  experts  were  about  1.5  km  in  length.  Figure  10  shows  that  most  of 
these  segments  have  a  velocity  difference  of  about  13  m/s,  and  in  Figure  11,  the  average 
shear  value  for  the  segment  was  about  9  m/s  km"*. 
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frequency 


segment  length  (km) 


Figure  9.  Rotation  Truth  Statistics:  Segment  Length 

The  histogram  above  shows  that  most  of  the  rotation  segments  found  by  experts  were  about  1.5  km  In 
length. 
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Figure  10.  Rotation  Truth  Statistics;  Velocity  Difference 

The  figure  show  that  most  of  the  rotation  segments  found  by  experts  have  a  velocity  difference  of  about 
13  mis. 
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6.  ANALYSIS  OF  RESULTS 


6.1  SCORING  RESULTS:  TEST  DATA 

Table  3  shows  the  results  of  13  test  runs  of  the  algorithm.  Note  that  that  delta v_thr, 
though  listed  as  one  of  the  parameters  changed,  is  not  included  in  the  parameter  list  in 
Table  2  because  its  value  was  set  to  10  m/s  at  the  beginning  of  the  testing  and  was  never 
changed  again.  The  values  in  the  third  column  in  Table  3  are  a  list  of  the  parameters  being 
varied  for  the  particular  test  for  each  row  in  the  table.  For  each  test,  the  POD,  PFD,  and 
DTR  were  calculated  and  are  also  shown  in  the  table. 


The  final  test  of  the  sequence  yielded  a  parameter  set  which  allowed  for  a  POD  of 
100  percent  for  both  Kansas  City  and  Huntsville,  with  an  average  of  45  percent  PFD. 

One  thing  that  can  be  seen  from  Table  3  is  that  as  one  test  led  to  the  next,  the  DTR 
first  decreased  for  both  Kansas  City  and  Huntsville,  reaching  their  low  points  on  Test  6, 
then  slightly  increased,  though  with  another  local  minimum  at  Test  12.  This  occurrence 
was  mostly  due  to  the  manipulation  of  the  area_thr,  which  eliminated  a  significant  number 
of  detections  (both  potential  hits  and  false)  when  set  to  a  value  greater  than  1.0  sq-km. 

Perhaps  the  most  noteworthy  result  is  the  complete  failure  of  the  tuning  of  these  particu¬ 
lar  parameters  in  some  cases  to  have  any  effect  on  the  algorithm’s  performance  for  Denver 
data.  The  reason  for  this  insensitivity  turned  out  to  be  the  extremely  low  signal-to-noise 
ratio  (SNR)  on  the  scans  under  test.  At  low  signal-to-noise-ratio,  there  are  many  data 
gaps  due  to  low  SNR  editing.  As  mentioned  in  section  5.3.1,  the  meteorologist  accounted 
for  this  and  still  found  a  truth  region,  but  the  algorithm  could  not  detect  rotation  regions 
with  the  large  number  of  data  gaps.  Figure  12  shows  a  typical  example  where  the  use 
of  SNR  threshold  eliminates  a  number  of  large  anomalous  velocities  at  the  expense  of  as¬ 
signing  bad  values  to  data  in  a  valid  rotation  region. 
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Table  3. 

Algorithm  improvement  Results  —  Test  Data,  by  Site 


test  # 

change  #* 

value  POD(%) 

PFD(%) 

DTR 

1 

2.1 

10,0.0 

70 

73 

8.5 

2 

1 

0.4 

70 

81 

7.8 

3 

1 

0.6 

70 

76 

7.2 

4 

1 

0.8 

90 

84 

5.7 

5 

3 

500 

80 

81 

4.4 

6 

4 

2.0 

60 

60 

2.0 

7 

4 

1.5 

70 

69 

2.9 

8 

5 

-1.3 

80 

66 

3.2 

9 

5.4 

-1 .3.2.0 

70 

60 

2.5 

10 

5.4 

-2.3.2.0 

80 

60 

2.7 

11 

3 

240 

80 

61 

3.4 

12 

3.5.4 

240,-2.3.1.5 

70 

57 

2.3 

13 

4.6 

1. 0.0.5 

100 

61 

3.8 

1 

2,1 

10,0.0 

0 

0 

0 

2 

1 

0.4 

0 

0 

0 

3 

1 

0.6 

0 

0 

0 

4 

1 

0.8 

0 

0 

0 

5 

3 

500 

0 

0 

0 

6 

4 

2.0 

0 

0 

0 

7 

4 

1.5 

0 

0 

0 

8 

5 

-1.3 

0 

0 

0 

9 

5.4 

-1 .3,2.0 

0 

0 

0 

10 

5.4 

-2.3,2.0 

0 

0 

0 

11 

3 

240 

0 

0 

0 

12 

3,5,4 

240,-2.3,1.5 

0 

0 

0 

13 

4,6 

1. 0,0.5 

0 

0 

0 
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Table  3.  (cent.) 

Algorithm  Improvement  Results  —  Test  Data,  by  Site 


test  # 

change  #* 

value  POD(%) 

PFD(%) 

DTR 

1 

2.1 

10.0.0 

100 

17 

1.3 

2 

1 

0.4 

100 

17 

1.3 

3 

1 

0.6 

100 

17 

1.3 

4 

1 

0.8 

100 

11 

1.0 

5 

3 

500 

100 

13 

0.89 

6 

4 

2.0 

89 

25 

0.44 

7 

4 

1.5 

89 

17 

0.67 

8 

5 

-1.3 

89 

17 

0.67 

9 

5.4 

-1.3.2.0 

89 

25 

0.44 

10 

5.4 

-2.3.2.0 

89 

25 

0.4 

11 

3 

240 

100 

10 

1.1 

12 

3.5.4 

240.-2.3,1.5 

89 

17 

0.67 

13 

4.6 

1. 0.0.5 

100 
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1.1 

*The  meaning  of  the  change  numbers  In  the  second  column  are  explained  below; 


m 

parameter  changed 

1 

shear_drop 

deltav_thr 

B 

range_thr 

D 

area_thr 

az_thr 

B 

min_segjen 
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Figure  12.  Azimuth  vs.  Velodty:  Signature  Analysis. 

The  plot  Is  showing  the  azimuthal  velocity  signatures  for  a  group  of  consecutive  range  gates.  The  top 
figure  shows  the  effect  on  the  data  after  SNR  thresholding  and  velocity  unfolding.  The  bottom  figure 
shows  date  that  has  not  been  pre-processed.  The  circle  Is  truth;  the  rectangles  vertically  overlapping 
the  signatures  ere  algorithm-generated  sheer  segments.  The  dashed  lines  are  +15  and  -15  mis  about 
the  velocity  axis  (solid  vertical  lines).  The  top  Is  labelled  In  km,  the  left  In  degrees. 


Kansas  City  data  seemed  to  be  most  sensitive  to  the  shear_drop  threshold  since  the 
high  environmental  wind  component  typical  of  the  geographical  area  caused  many  false 
segments  to  be  identified.  By  allowing  more  flexibility  in  the  requirements  for  growing 
a  segment,  the  POD  was  improved  for  Kansas  City,  although  its  PFD  then  suffered.  This 
effect  is  shown  in  the  color  radar  plot  in  Figure  13,  where  the  red  rectangle  is  an  algorithm 
bounding  box  overlaid  on  the  radar  data.  This  box  is  a  false  detection,  where  an  intermedi¬ 
ate  version  of  the  algorithm  mistakenly  detects  the  gradual  change  in  wind  velocity  over 
the  broad  region  of  the  scan  as  a  genuine  azimuthal  shear. 

One  of  the  effects  of  using  the  final  combination  of  all  these  thresholds  is  illustrated 
in  Figure  14  in  which  the  shear  segments  as  well  as  algorithm  bounding  boxes  are  pictured. 
In  this  case,  shorter  segments  led  to  a  significant  decrease  in  the  PFD  by  giving  rise  to 
regions  with  insufficient  area  to  be  considered  as  valid  output  data. 

Overall,  the  number  of  valid  segments  found  was  quite  a  bit  higher  with  the  final  version 
of  the  algorithm.  For  Test  1,  there  were  4258  unused  segments  (segments  whose  associated 
region  was  invalidated  by  thresholding),  and  709  belonging  to  64  valid  regions.  For  the 
final  test,  there  were  7177  unused  segments,  with  824  belonging  to  68  valid  regions.  This 
was  to  be  expected  because  the  goal  was  for  a  coarser,  less  sensitive,  and  thereby  more 
effective  detection  capability,  which  is  made  possible  by  generating  more  numerous,  though 
shorter,  segments. 

At  least  for  higher  reflectivity  environments  such  as  Kansas  City  and  Huntsville,  it  can 
be  said  that  the  algorithm  performance  was  improved  in  general  by  tightening  the  range 
association  criterion  from  1  km  to  240  m  and  relaxing  the  requirement  of  a  1  degree  in 
azimuth  overlap  to  allowing  a  gap  of  2.3  degrees.  The  fact  that  Denver’s  microbursts  tended 
to  have  generally  lower  differential  velocities,  as  well  as  less  precipitation,  [9]  made  detec¬ 
tion  more  difficult  there.  In  addition,  reducing  the  segment  length  threshold,  min_seg_len, 
made  a  dramatic  difference  in  performance  for  Kansas  City  and  Huntsville,  improving  the 
POD  by  as  much  as  30  percent,  while  increasing  the  PFD  only  a  small  amount.  Shortening 
the  segment  length  threshold  from  1  km  to  0.5  km  works  in  conjunction  with  the  association 
thresholds  to  allow  short  segments  to  be  grouped  together,  thereby  allowing  the  algorithm 
to  be  successful  in  a  noisy  velocity  environment. 
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Figure  13.  Resampled  radar  data  with  overlays-  False  detections 

A  Terminal  Doppler  Weather  Radar  image  from  Kansas  City,  MO,  on  July  30,  1989,  at  06:54:56  UT 
at  an  elevation  angle  of  4.5  degrees.  The  color  bars  at  the  bottom  shows  the  velocity  mapping  in  m/s. 
Azimuth  lines  are  5  degrees  apart,  range-rings  are  5  km  apart  Red  rectangle  is  a  false  rotation  detection. 
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Figure  14.  Comparison  of  results — First  Parameter  Set  to  Last  Parameter  Set 


One  of  the  effects  of  using  the  final  combination  of  all  these  thresholds  Is  Illustrated  above.  Pictured 
here  are  the  shear  segments,  as  well  as  algorithm  bounding  boxes,  which  are  all  false  detections.  The 
top  plot  shows  the  output  from  the  algorithm  using  the  first  parameter  set;  the  bottom  shows  the  output 
from  the  last  parameter  set.  Note  the  shorter  segments  and  reduction  In  number  of  regions  In  the  bottom 


6.2  SCORING  RESULTS— CASE  DATA 


Once  it  became  apparent  as  to  which  logic  and  parameter  changes  to  incorporate  into 
a  final  version,  called  Version  1.5,  the  algorithm  was  run  on  the  large  data  set.  The  com¬ 
plete  set  of  results,  arranged  by  case  data  and  location,  are  given  in  Table  4.  A  quick 
overall  comparison  of  Versions  1.4  and  1.5  is  shown  below. 


POD 

PFD 

DTR 

POD 

PFD 

DTR 

wet 

89 

54 

2.9 

wet 

84 

37 

2.1 

dry 

59 

60 

1.9 

dry 

40 

64 

1.5 

Version  1.4  Version  1.5 


wet  =  Kansas  City  and  Huntsville 
dry  =  Denver 

It  can  be  seen  that  the  algorithm  has  an  obvious  sensitivity  to  the  geographical  region 
from  which  radar  data  is  obtained,  and  so  parameter  tuning  to  improve  algorithni  perform¬ 
ance  must  be  done  on  a  regional  basis.  This  conclusion  was  also  arrived  at  from  the  discri¬ 
minant  analysis,  which  will  be  seen  in  section  8.1.  Therefore,  it  was  most  sensible  to  sepa¬ 
rate  the  final  results  into  high  reflectivity  (wet)  and  low  reflectivity  (dry)  region 
performance.  High  reflectivity  is  considered  to  be  above  45  dbZ,  and  low  reflectivity  is 
15  dbZ.  The  performance  degradation  seen  for  the  low  reflectivity  results  was  because 
of  the  method  of  parameter  tuning,  and  this  will  be  discussed  further  in  section  6.4. 

While  Version  1.4  was  preferred  for  Huntsville  and  one  case  date  (July  1,  189)  in  Kan¬ 
sas  City,  Version  1.5  was  better  for  the  other  case  date  in  Kansas  City,  July  30,  1989. 

Overall,  the  results  are  that  the  variation  of  parameters  can  significantly  decrease  the 
DTR,  which  is  desirable,  and  this  will  also  decrease  the  PFD,  but  with  a  small  decrease 
of  the  POD.  At  most,  the  POD  was  reduced  by  nine  percent  for  Kansas  City  and  Huntsville. 
However,  roughly  one-third  of  the  detections  were  lost  for  the  Denver  case,  with  the  result 
that  the  POD  dropped  by  19  percentage  points. 

The  reduction  in  PFD  was  most  significant  for  the  Kansas  City  data,  which  could  be 
made  as  low  as  15  percent.  This  reduction  in  PFD  was  achieved  with  a  reduction  of  only 
three  percent  in  POD. 
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Table  4. 


Final  Scoring  Results  —  Case  Data,  by  Site 


-  Kansas  City  - 

Version  date _ truth _ hHs _ regions  region  hits  POD _ PFD _ DTR 


1.4 

jul01_89 

428 

354 

652 

461 

83 

30 

1.5 

using  1 

.0  km  min.  ait 

.,1.0  sq-km  area 

1.5 

jul01_89 

414 

333 

530 

448 

80 

15 

1.3 

using  0 

.5  km  min.  alt 

.,1.0  sq-km  area 

1.5 

jul01_89 

414 

338 

563 

450 

82 

20 

1.4 

using  1 

.0  km  min.  alt 

.,2.0  sq-km  area 

1.5 

jul01_89 

414 

333 

548 

448 

80 

18 

1.3 

1.4 

jul30_89 

304 

292 

1473 

513 

96 

65 

4.8 

using  1 

.0  km  min.  alt 

.,1.0  sq-km  area 

1.5 

jul30_89 

303 

268 

752 

416 

88 

45 

2.5 

using  0 

.5  km  min.  alt 

.,1.0  sq-km  area 

1.5 

jul30_89 

303 

271 

911 

485 

89 

53 

3.0 

using  1 

.0  km  min.  alt 

.,2.0  sq-km  area 

1.5 

jul30_89 

303 

268 

752 

416 

88 

45 

2.5 

- — - - - Deitver - — - - - 

VargiM _ sialfi _ truth _ bits _ regions  region  hits  POD  PFD 


1.4  sep02_87  140  82 

272 

107 

59 

60 

1.9 

using  1.0  km  min.  alt.,  1.0  sq-km  area 

1.5  sep02_87  134  48 

129 

64 

36 

50 

0.96 

using  0.5  km  min.  alt., 1.0  sq-km  area 

1.5  sep02_87  134  54 

206 

132 

40 

64 

1.5 

using  1.0  km  min.  alt., 2.0  sq-km  area 

1.5  sep02_87  134  48 

129 

64 

36 

50 

0.96 

- Huntsville - 

Version  date  truth _ bllfi _ CfiOifiOS _ rMion  .hits  JOB _ EESL 


1.4 

jul25_86 

42 

41 

134 

65 

98 

51 

using  1 

.0  km  min.  alt 

.,1.0  sq- 

■km  area 

1.5 

jul25_86 

42 

38 

99 

53 

90 

46 

using  0 

.5  km  min.  alt 

.,1.0  sq- 

km  area 

1.5 

jul25_86 

42 

38 

103 

53 

90 

49 

using  1 

.0  km  min.  alt 

.,2.0  sq- 

■km  area 

1.5 

jul25_86 

42 

38 

99 

53 

90 

46 

Three  different  types  of  scoring  was  performed  on  Version  1.5,  as  shown  in  Table  4. 
There  are  two  thresholds  in  the  rotation  algorithm  that  are  set  to  different  values  than  the 
corresponding  values  in  the  microburst  recognition  system.  These  are  the  minimum  altitude 
threshold  and  the  minimum  area  threshold,  which  are  set  to  0.5  km  and  1.0  sq-km,  respec¬ 
tively,  in  the  rotation  algorithm  and  1.0  km  and  2.0  sq-km  in  the  microburst  recognition 
system.  Since  ASP  has  the  ability  to  vary  these  parameters  for  scoring  purposes,  it  was 
interesting  to  see  the  difference  in  performance  statistics. 

It  may  be  noted  that  there  are  slight  differences  in  the  number  of  truth  observations 
within  the  same  case  data  when  run  on  different  versions.  This  is  because  of  a  scoring 
philosophy  which  states  that  truth  must  reside  within  the  same  35  km  limit  as  regions  in 
order  to  be  used  for  scoring.  If  not,  it  is  eliminated  from  the  scoring  process  and  will  not 
be  counted  in  the  total  number  of  truth  observations.  If  a  region  hits  it,  however,  then 
it  is  included  in  the  total,  and  the  detections  will  be  counted  as  hits.  Therefore,  if  a  particular 
run  of  the  algorithm  fails  to  produce  a  particular  region,  then  it  is  possible  that  the  truth 
observation  corresponding  to  it  will  be  ignored  in  the  POD  calculation  if  it  is  outside  the 
35  km  radius  of  the  scoring  region. 


6.3  EVENT  SCORING  RESULTS 

Scoring  the  algorithm  on  an  event  basis  resulted  in  an  event  recognition  rate  of  94 
percent  for  higher  reflectivity  regions  and  78  percent  for  low  reflectivity  regions.  There 
was  no  change  in  the  RR  for  Kansas  City  and  Huntsville  between  Versions  1.4  and  1.5, 
but  for  Denver,  the  RR  dropped  by  11  percentage  points.  The  event  scoring  is  given  in 
Table  5,  showing  also  the  number  of  events  having  early  or  late  detections.  One  would 
expect  these  results  to  be  analogous  to  the  observation  scoring  in  the  previous  section. 


Table  5.  Event  Scoring  Results 


Version  date 


JaiSb- 


MS- 


raoions _ region -hits.  PQP- 


DTR 


1.4 

jul01_89 

8 

r 

1.5 

jul01_89 

8 

1.4 

jul30_89 

6 

1.5 

jul30_89 

6 

_ 1 

1.4 

sep02_87 

9 

1.5 

sep02_87 

9 

1.4 

1.5 


jul25_86 

jul25_86 


3 

3 


Kansas  City 


Denver 


Huntsville  ■ 


7 

7 

6 

6 


8 

7 


3 

3 


7 

7 

6 

5 


6 

4 


3 

3 


7 

7 

6 

5 


8 

4 


3 

3 


88 

88 

100 

100 


89 

78 


100 

100 
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6.4  DISCUSSION 


The  first  issue  to  be  addressed  is  the  apparent  discrepancy  in  performance  between 
the  results  for  test  data  and  for  case  data.  Some  difference  was  expected,  however,  given 
the  difficulty  of  choosing  a  generic  set  of  test  scans.  In  addition,  the  test  data  was  biased 
toward  producing  false  rotation  regions  since  the  major  goal  in  improving  the  algorithm 
was  to  reduce  the  PFD.  Before  the  testing  commenced,  however,  it  was  not  known  that 
the  Denver  data  would  essentially  drop  out  of  the  evaluation,  with  its  especially  difficult 
representative  scans.  It  was  also  discovered  later  how  sensitive  the  parameter  tuning  was 
to  the  region  associated  with  a  data  set.  Therefore,  the  ultimate  parameter  settings  were 
effectively  geared  toward  performance  under  conditions  of  high  reflectivity. 

It  is  clear,  however,  that  although  the  POD  was  greatly  improved  for  the  twenty-scan- 
test-data  set,  it  was  made  somewhat  worse  for  the  case  data  set  of  588  scans.  This  was 
because  of  the  overall  tendency  of  the  final  parameter  set  to  allow  for  shorter  segments. 
Since  the  area  of  a  region  is  calculated  by  totaling  the  areas  of  all  its  constituent  segments, 
the  important  effect  of  having  shorter  segments  was  to  reduce  the  overall  area  for  a  candi¬ 
date  region.  This  means  that  the  test  data  scans,  which  happened  to  cause  the  algorithm 
to  detect  a  large  numbers  of  segments,  would  now  declare  as  valid  a  region  it  had  previously 
discarded  because  it  was  too  large.  Therefore,  what  would  have  been  a  miss  was  now  a 
hit.  These  regions  apparently  had  areas  well  in  excess  of  the  a.rea  criteria.  For  the  case 
data,  the  characteristic  of  shorter  segments  statistically  had  more  of  an  effect  on  reducing 
the  number  of  regions  generated.  The  average  area  for  rotation  regions  identified  by  Ver¬ 
sion  1.4  is  3.57  sq-km,  whereas  for  Version  1.5  it  is  2.5  sq-km,  a  decrease  of  32  percent. 
Therefore,  if  a  region  detected  by  Version  1.4  was  small  enough  to  begin  with,  a  reduction 
in  its  area  by  an  average  of  32  percent  would  lead  to  its  invalidation  by  Version  1.5,  and 
thus  the  POD  was  reduced. 

A  number  of  general  observations  can  be  made  here  with  respect  to  the  scoring  results. 
One  is  that  the  misses  were  generally  caused  by  the  absence  of  a  rotation  region  declaration, 
as  opposed  to  the  declaration  of  region  that  was  too  distant  from  a  truth  region.  That  is, 
there  were  no  spatial  near-misses.  If  the  algorithm  missed  a  detection,  it  was  because 
of  one  of  two  reasons.  First,  if  too  few  segments  were  generated  for  the  region,  usually 
due  to  low  signal-to-noise  ratio,  the  area  of  the  resulting  region  would  be  too  low  to  be 
considered  valid,  as  with  Denver  data.  Second,  if  too  many  segments  were  generated,  as 
with  strong  environmental  winds,  the  region  would  be  eliminated  for  having  too  large  an 
area. 

A  strong  environmental  wind  could  introduce  false  detections  because  the  algorithm 
detects  only  monotonically  increasing  (or  decreasing)  velocities,  not  necessarily  passing 
through  zero  from  negative  to  positive.  Inhomogeneities  in  the  windfield  would  cause  the 
algorithm  to  interpret  changes  in  the  wind  speed  as  the  endpoints  of  shear  segments.  In 
Figure  15,  the  area  within  the  circle  represents  part  of  a  radar  scan,  centered  on  the  radar. 
A  horizontal  eastward  wind  is  shown  as  dotted  lines.  The  length  of  the  solid  arrows  represent 
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the  magnitude  of  the  the  radial  components  of  a  the  wind  as  seen  by  the  radar.  The  darker 
arrows  indicate  radial  velocities  away  from  the  radar,  and  the  lighter  arrows  are  velocities 
toward  the  radar.  At  due  north  and  south,  the  radial  component  of  the  wind  is  zero,  while 
at  east  and  west  the  radial  component  of  the  environmental  wind  is  a  maximum.  If  the 
area  of  the  horizontal  wind  were  within  the  size  parameter  of  the  algorithm  and  were  of 
sufficient  intensity,  this  would  result  in  the  false  detection  of  a  rotation  region.  The  box 
represents  such  a  false  detection. 

A  second  related  point  is  how  relatively  infrequently  detections  were  early  or  late  rather 
than  false.  This  is  illustrated  in  Figure  16,  where  the  number  of  each  type  of  detection 
is  plotted  opposite  the  cases  label.  The  points  in  this  diagram  were  sorted  in  ascending 
order  of  the  number  of  false  detections  to  produce  a  graph  that  would  clearly  show  the 
relationships.  One  interesting  observation  is  that  when  the  total  number  of  regions  was 
relatively  small,  there  were  roughly  as  many  early  and  late  detections  as  false  detections. 
When  the  total  number  was  large,  however,  there  were  many  more  false  detections  than 
early  or  late  detections.  The  reason  for  this  is  similar  to  the  observation  above  regarding 
misses,  since  early  and  late  detections  have  a  close  spatial  relationship  to  some  truth  obser¬ 
vation,  being  displaced  from  it  only  in  time.  This  happens  when  there  are  slightly  different 
data  conditions  on  successive  scans,  making  it  possible  for  the  overlap  to  occur  with  truth 
on  a  scan  before  or  after  the  algorithm’s  declaration,  though  it  misses  the  current  scan. 
TY  .e  ca-e  of  a  detection  being  declared  early  or  late,  for  the  reasons  given  in  the  preceding 
paiig^aph,  should  therefore  occur  infrequently  with  respect  to  the  total  number  of  detec¬ 
tions.  By  noting  the  different  scales  on  the  y-axes  of  the  figure,  one  can  also  see  the  differ¬ 
ence  in  overall  region  production  in  Version  1.4  as  compared  to  Version  1.5. 

Because  the  design  of  the  current  TDWR  testbed  real  time  feature  extraction  system 
is  to  process  one  radial  at  a  time,  it  was  not  possible  for  the  rotation  algorithm  to  have 
an  entire  scan’s  reflectivity  information,  so  it  could  not  make  any  association  with  storm 
cells  identified  in  the  reflectivity  image.  The  microburst  algorithm,  on  the  other  hand,  can 
discard  rotation  regions  that  are  not  spatially  near  a  reflectivity  structure,  but  the  result 
is  that  the  rotation  algorithm  appears  to  have  a  high  PFD. 
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Figure  15.  How  an  Environmental  Wind  Can  Causa  Falsa  Detections. 

The  area  within  the  circle  represents  part  of  a  radar  scan,  centered  on  the  radar.  A  horizontal  eastward 
wind  Is  shown  as  dotted  lines.  The  length  of  the  solid  arrows  represent  the  magnitude  of  the  the  radial 
components  of  the  wind  as  seen  by  the  radar.  The  darker  arrows  Indicate  radial  velocities  away  from 
the  radar,  and  the  lighter  arrows  are  velocities  toward  the  radar.  At  due  north  and  south,  the  radial  compo¬ 
nent  of  the  wind  Is  zero,  while  at  east  and  west  the  radial  component  of  the  environmental  wind  Is  a 
maximum.  If  the  area  of  the  horizontal  wind  were  within  the  size  parameter  of  the  algorithm  and  were 
of  sufficient  Intensity,  this  would  result  In  the  false  detection  of  a  rotation  region.  The  box  represents 
such  a  false  detection. 
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Figure  16.  Relative  Distribution  of  Detections. 

Detections  were  more  likely  to  be  classified  as  false  than  as  early  or  late  for  the  nine  different  cases  plotted. 
The  point  symbols  are:  ■  false,  A  early,  x  late. 


In  retrospect,  if  an  entire  scan’s  worth  of  data  were  to  be  stored,  then  two-dimensional 
image  processing  techniques  could  be  performed.  The  advantage  is  that  the  two-dimension¬ 
al  techniques  are  more  well  known  that  the  one-dimensional  approach  being  taken  here. 
Furthermore,  using  such  a  technique  would  produce  a  set  of  rotation  regions  that  could 
be  retained  for  correlation  with  regions  found  in  subsequent  scans,  resulting  in  a  measure 
of  the  persistence  of  a  detection.  Even  with  the  data  buffered,  given  the  current  scan  strate¬ 
gy  stated  in  section  2.1.3,  time  correlation  would  be  restricted  to  a  2.5  minute  sampling 
of  a  particular  altitude,  which  is  less  than  satisfactory  for  a  short  term  event  such  a  micro¬ 
burst. 

Given  that  further  validation  and  rejection  of  rotation  regions  is  done  at  the  microburst 
algorithm  level,  which  is  subsequent  to  the  feature  extraction  phase  (see  section  2.1.2), 
there  is  no  good  criteria  at  this  time  for  acceptable  POD  and  PFD  for  rotation  detection. 

Lastly,  on  the  whole,  the  process  of  improving  the  rotation  algorithm  provided  valuable 
insights  into  its  weaknesses,  strengths,  and  sensitivities  on  a  per-locale  basis.  The  algorithm 
can  be  successful  only  if  its  parameters  have  been  tuned  to  the  environment  in  which  it 
will  be  used.  The  algorithm  performed  best  in  high  reflectivity  environments  because  this 
provided  a  high  SNR,  and  therefore  a  smoother  signature,  such  as  those  shown  in  Figure 
17.  Furthermore,  it  was  found  to  be  impossible,  in  practice,  to  raise  the  POD  without  also 
raising  the  PFD,  given  the  amount  of  information  the  algorithm  has  at  its  disposal. 
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FL-2  INyll/vcDlestrot/cfl  7-25-86  21:59:57 
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21800 


21680 


21560 


21440 


Figure  17.  Sample  Rotation  Signatures:  Huntsville,  AL. 

Plots  of  velocity  as  a  function  of  azimuth  for  four  consecutive  range  values  for  test  data  from  Huntsville, 
AL  on  July  25, 1986,  at  21:59:57  UT.  Scan  direction  Is  counter-dockv/ise,  which  accounts  for  the  ranges 
In  reverse  order  vertically.  Long  horizontal  H-shaped  bars  represent  algorithm  shear  segments,  which 
delineate  the  sloping  velocity  signatures.  The  azimuth  values  are  In  degrees,  the  velodty  axis  Is  meterslse- 
cond,  and  the  range  values  are  In  meters. 
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7.  SIMULATION  STUDIES 


7.1  INTRODUCTION 

An  alternative  approach  to  analyzing  the  performance  of  the  rotation  algorithm  is  to 
simplify  its  data  input,  which  would  therefore  alleviate  the  task  of  finding  representative 
test  data.  This  has  been  accomplished  by  utilizing  a  program  that  can  generate  simulated 
rotation  regions.  The  additional  advantages  to  this  are  obvious — the  elimination  of  data 
contaminants  such  as  clutter,  velocity  folding,  and  some  inherent  ambiguities  of  the  scoring 
process,  as  well  as  achieving  greater  clarity  (by  definition)  of  the  truth.  All  of  this  contrib¬ 
utes  to  a  higher  confidence  factor  for  the  conclusions  found. 

The  previous  notwithstanding,  the  main  reason  to  use  a  simulated  data  set  is  to  verify 
the  degree  to  which  the  algorithm  “perfectly”  detects  a  “perfect”  rotation  region.  Perfection 
in  this  case  is  defined  as  a  POD  of  100  percent  and  a  PFD  of  0. 

Six  different  data  sets  were  created  for  use  with  three  experimental  versions  of  the 
algorithm,  each  with  two  different  parameter  sets.  This  chapter  contains  an  analysis  of  these 
tests  and  offers  some  solutions. 

7.2  PARAMETER  DIFFERENCES 

The  parameter  sets,  called  in  this  chapter  site  and  final,  differed  as  shown  in  Table 
6.  Any  other  parameters  involved  were  the  same  in  both  parameter  sets.  The  nature  of 
the  parameters  listed  in  the  table  are  explained  in  Section  4.4.2. 

7.3  ALGORITHM  VERSION  DIFFERENCES 

The  three  versions  of  the  algorithm  used  for  this  study  will  be  referred  to  here  as  A, 
B,  and  C  for  simplicity.  The  differences  between  them  are  briefly  outlined  below; 

•  A  is  the  current  operational  version  running  at  the  testbed  and  is  implemented 
as  specified  in  the  AEL.  It  shares  a  feature  with  Version  B,  described  below, 
in  that  it  will  continue  a  segment  even  if  velocities  are  constant. 

•  B  has  the  shear  validation  test  described  in  section  4.4.1,  a  type  of  segment  crop¬ 
ping  technique  which  allows  a  shear  segment  to  grow  only  if  its  current  shear 
value  (equal  to  the  segment’s  differential  velocity  divided  by  its  length)  will  not 
decrease  with  the  addition  of  the  new  azimuth  gate.  This  is  used  to  prevent  a 
segment  from  becoming  too  long,  even  though  the  velocity  difference  between 
its  endpoints  may  be  of  sufficient  magnitude  to  pass  a  velocity  validation  test. 
The  main  difference  between  B  and  C  is  that  B  will  continue  to  build  a  segment 
even  if  the  velocities  are  changing  only  slightly. 

•  C  has  the  three-radial  buffering  as  described  in  Section  4.4.1,  intended  to  accom¬ 
plish  some  data  smoothing.  This  procedure  has  the  effect  of  smoothing  the  data 
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by  “blurring”  the  data  points.  In  addition,  this  version  will  end  any  segment 
it  may  have  started  if  the  difference  between  neighboring  velocities  is  small 
enough  (i.e.,  an  absolute  value  less  than  or  equal  to  vel_thr,  which  is  currently 
set  to  0). 

7.4  TEST  SUITE 

The  data  for  this  study  was  produced  by  a  program  that  synthesized  radar  image  data 
according  to  a  set  of  variable  input  parameters.  The  image  contained  a  set  of  azimuthally 
sheared  velocity  points  which  form  “perfect”  rotation  regions,  such  that  its  velocity  vs.  azi¬ 
muth  profile  would  appear  as  in  Figure  4b.  All  of  the  simulated  regions  were  elliptical, 
and  centered  at  range  and  azimuth  of  10  km,  45  degrees,  measured  with  respect  to  the 
radar.  The  sheared  region  was  oriented  at  135  degrees  from  north,  meaning  that  the  velocity 
shear  was  aligned  at  135  degrees.  Those  regions  whose  eccentricity  was  1  as  listed  in  Table 
6  were  symmetrical;  the  major  axis  =  minor  axis  =  2  km.  For  those  regions  whose  eccentric¬ 
ity  was  listed  as  0.5,  the  major  axis,  parallel  to  the  azimuth  axis,  was  set  to 
2  km,  and  the  the  minor  axis,  parallel  to  the  range  axis,  was  1  km. 

For  all  of  the  simulated  regions,  the  velocity  in  the  area  surrounding  the  sheared  region 
was  initialized  to  0  m/s.  An  example  of  a  simulated  rotation  region  is  shown  in  Figure 
18  and  the  region  attributes  are  summarized  in  Table  7. 


Table  6. 

Differences  in  the  Parameter  Sets 


Parameter 

SITE 

FINAL 

rangejhr 

1000  m 

240  m 

azjhr 

0  degrees 

-2.3  degrees 

mln_segjen 

1.0  km 

0.5  km 

deltav 

7  m/s 

10  m/s 

Table  7. 


Characteristics  of  the  Simulated  Data  Test  Suite 


Case  # 

Velocity  Differential 
(m/s) 

Eccentricity 

1 

30 

1 

II 

30 

0.5 

III 

20 

1 

iV 

20 

0.5 

V 

12 

1 

VI 

12 

0.5 
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Azimuth  (degrees) 


Range  (km) 


l,  :.l  St  -I  It.  a  :l.1  -l.>  ■«  St  IS'  1 .  11  '  il 


Figure  18.  Polar  Coordinate  Plot  of  Simulated  Rotation  Region 

Warm  colors  (red,  orange)  are  representative  of  velocities  receding  from  the  radar,  and  cool  colors  (blue, 
green)  represent  approaching  velocities.  Top  axis  Is  range,  labelled  In  km,  left  axis  Is  azimuth  In  degrees. 
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7.5  TEST  RESULTS 


Because  of  the  accuracy  of  truth  in  the  case  of  simulated  data,  it  was  possible  to  evaluate 
the  performance  of  the  three  algorithms  on  a  different  level  than  with  real  data.  The  scoring 
was  performed  on  a  point-by-point  basis,  where  each  velocity  point  identified  by  an 
algorithm  as  belonging  to  a  valid  segment  was  judged  as  either  inside  the  truth  region 
(valid),  or  outside  the  region  (false).  This  gave  a  much  finer  resolution  for  the  calculation 
of  the  POD  and  the  PFD.  The  number  of  algorithm  points  inside  the  simulated  region 
boundary  divided  by  the  number  of  truth  points  in  this  region  gave  the  POD.  The  number 
of  algorithm  points  outside  the  truth  region  divided  by  the  total  number  of  points  generated 
by  the  algorithm  gave  the  PFD. 

Table  8  summarizes  the  performance  of  the  three  algorithms  with  the  simulated  data. 
Figure  19  shows  a  sample  listing  of  the  segments  found  by  algorithm  A  using  the  final 
parameters  for  region  VI. 


Table  8. 

Simulation  Test  Results:  POD  and  PFD  by  Case 


Version/parameter  set 

CASE 

T 

A/Site  A/finai  B/site  B/final  C/site  C/final 

POD 

1 

100 

100 

97 

97 

98 

98 

II 

98 

100 

90 

90 

89 

89 

■■1 

100 

100 

98 

98 

100 

100 

IV 

100 

100 

100 

100 

99 

100 

V 

100 

100 

99 

99 

99 

99 

VI 

100 

100 

99 

99 

100 

100 

PFD 

1 

15 

36 

12 

25 

13 

14 

il 

11 

23 

6 

9 

7 

8 

20 

38 

18 

22 

17 

19 

IV 

22 

32 

11 

11 

10 

12 

V 

32 

42 

27 

27 

29 

29 

VI 

31 

41 

25 

25 

24 

24 
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STRTAZ 

STPAZ 

DV 

SHR 

RNG 

MINV 

MAXV 

40.0 

50.0 

-6.6 

4.1 

9120 

3.3 

-3.3 

40.0 

50.0 

-8.4 

5.2 

9240 

4.2 

-4.2 

40.0 

50.0 

-9.2 

5.7 

9360 

4.6 

-4.6 

40.0 

50.0  -10.4 

6.3 

9480 

5.2 

-5.2 

40.0 

50.0  -11.6 

6.5 

10200 

5.8 

-5.8 

40.0 

50.0  -10.6 

5.8 

10440 

5.3 

-5.3 

43.0 

50.0 

-7.9 

6.1 

10560 

3.1 

-4.8 

40.0 

50.0 

-8.9 

4.8 

10680 

4.4 

-4.5 

40.0 

50.0 

-7.4 

3.9 

10800 

3.7 

-3.7 

40.0 

50.0 

-5.5 

2.9 

10920 

2.7 

-2.8 

39.0 

51.0  -11.2 

5.6 

9600 

5.6 

-5.6 

40.0 

51.0  -11.6 

6.2 

9720 

5.8 

-5.8 

39.0 

51.0  -11.8 

5.7 

9840 

5.9 

-5.9 

39.0 

51.0  -11.9 

5.7 

9960 

5.9 

-6.0 

39.0 

51.0  -11.8 

5.6 

10080 

5.9 

-5.9 

39.0 

51.0  -11.2 

5.2 

10320 

5.6 

-5.6 

Figure  19.  Sample  Listing  of  Segments  Found  for  a  Simulated  Region. 

In  the  listing,  STRTAZ  Is  the  starting  azimuth  of  the  segment  In  degrees,  STPAZ  is  the  ending  azimuth, 
DV  Is  the  velocity  differential  in  mis,  SHR  is  the  shear  in  misikm,  RNG  Is  the  range  of  the  segment  from 
the  radar  in  meters,  MINV  is  the  minimum  velocity  endpoint  of  the  segment  In  mis,  and  MAXV  is  the 
maximum  velocity  endpoint. 


According  to  Table  8,  the  best  detection  performance  overall  was  that  of  version  A, 
though  its  false  detection  rate  was  somewhat  higher  using  the  final  parameter  set.  None 
of  the  algorithms  performed  absolutely  perfectly,  which  would  mean  every  single  point  in 
its  proper  place,  but  the  POD  never  went  below  89  percent,  and  rarely  below  98  percent. 
The  PFD  was  generally  lower  for  versions  B  and  C. 

All  of  the  algorithms  generated  the  most  false  detections  for  the  weaker  regions  (V 
and  VI).  The  algorithms  had  the  best  detection  performance  for  the  moderate  regions  (HI 
and  IV).  The  lowest  PPTD’s  in  general  were  for  the  strongest  regions  (I  and  A). 

In  all  cases,  the  final  parameters  generated  more  detections  than  the  site  parameters, 
which  resulted  often  in  an  increased  POD  as  well  as  a  higher  rate  of  false  detections. 

Varying  the  eccentricity  of  the  simulated  region  in  cases  n  and  IV  appeared  to  have 
somewhat  lowered  the  number  of  false  detections  in  general.  For  algorithm  B  and  case 
n,  the  asymmetrical  region  in  combination  with  the  higher  shear  caused  the  POD  to  drop 
10  percent,  and  since  the  PFD  was  also  lowered,  the  algorithm  was  producing  fewer  detec¬ 
tions  in  general.  Algorithm  B’s  behavior  for  case  IV,  however,  is  quite  different,  and  so 
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there  is  no  clear  correlation  here.  In  fact,  since  both  regions  caused  B  to  detect  the  same 
number  of  segments,  the  difference  is  only  in  the  length  of  the  segments.  In  any  case, 
more  study  is  needed  to  determine  the  effect  of  asymmetry  on  detection  of  rotation. 

The  performance  of  the  algorithms  was  also  evaluated  on  the  basis  of  the  number  of 
valid  segments  (belonging  to  an  output  region)  found  by  the  algorithms  as  compared  with 
the  number  of  truth  segments  in  the  simulated  regions,  and  the  results  are  given  in  Table 
9.  It  can  be  seen  from  Table  9  that  it  is  not  sufficient  to  evaluate  performance  on  the 
basis  of  the  number  of  valid  segments,  since  in  every  case  the  POD  would  be  100  percent. 


In  general,  version  B  produced  fewer  segments  than  the  other  algorithms.  In  general 
the  algorithms  produced  more  segments  than  in  the  truth  regions,  and  these  were  always 
along  the  edges  of  the  region,  where  it  is  a  judgment  call  by  the  truther  what  are  the  “true” 
boundaries  of  the  region. 


Table  9. 


Number  of  Valid  Segments  for  Algorithms  and  Truth  by  Case 


case 

A/site 

A/final 

B/site 

B/final 

C/Site 

C/final 

Truth 

1 

27 

28 

27 

28 

27 

28 

25 

II 

20 

20 

17 

17 

17 

18 

19 

III 

25 

26 

25 

26 

25 

26 

23 

IV 

18 

18 

17 

17 

17 

18 

17 

V 

21 

21 

20 

20 

21 

21 

17 

VI 

16 

16 

16 

16 

16 

16 

14 

total 

127 

129 

122 

124 

123 

127 

115 
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7.6  CONCLUSIONS 


7.6.1  For  the  Study 

The  algorithms  in  general  performed  better  on  the  the  stronger  rotation  regions,  espe¬ 
cially  with  respect  to  the  frequency  of  false  detections.  Although  the  results  of  this  study 
are  not  dramatic,  they  do  show  that  the  algorithms  can  detect  a  perfect  rotation,  but  the 
region  is  over-detected. 

The  best  performance  on  perfect  data  was  that  of  version  A,  using  the  site  parameter 
set.  This  is  encouraging,  because  it  verifies  the  correcmess  of  the  original  design. 

7.6.2  In  General 

The  simulation  study  provided  information  that  was  actually  more  useful  for  decreasing 
the  PFD  (than  for  increasing  the  POD)  by  making  it  possible  to  isolate  the  sources  of  algo¬ 
rithm  sensitivities  that  were  not  obvious  from  field  data  analysis. 

The  simulation  was  helpful  also  because  it  revealed  the  importance  of  the  minimum 
segment  length  threshold.  Earlier  in  the  study,  the  tests  were  conducted  in  such  a  way 
that  a  particular  parameter  setting  was  preventing  the  algorithm  from  ever  detecting  the 
rotation  region.  The  parameter  was  modified,  and  the  tests  were  run  again  on  regions  large 
enough  to  be  detected  properly. 


8.  FUTURE  WORK 


8.1  DISCRIMINANT  ANALYSIS 

Since  it  would  be  beyond  the  scope  of  this  report  to  provide  a  tutorial  on  discriminant 
analysis,  it  suffices  to  say  that  such  an  analysis  is  one  that  involves  the  problem  of  the 
classification  of  observations  based  on  a  set  of  measured  features  [10].  As  it  applies  to 
improving  the  rotation  algorithm,  this  would  be  the  task  of  classifying  detections  as  either 
“valid”  or  “false.”  This  involves  the  specification  of  a  distribution  model  through  estima¬ 
tion  of  the  mean  feature  vector  and  covariance  matrix  for  each  class.  The  covariance  matrix 
shows  the  relationships  and  dependencies  between  the  various  features  of  a  rotation  region 
and  their  relative  importance  in  classifying  the  detection.  If  this  discriminant  function 
were  to  be  implemented  in  the  rotation  algorithm,  then  when  a  region  had  passed  all  its 
current  validation  thresholds,  the  probability  of  its  being  a  valid  detection  would  be  calcu¬ 
lated  from  the  discrimination  function  metric.  This,  as  a  final  test,  would  invalidate  those 
regions  it  judged  to  be  ultimately  false  detections  and  keep  only  those  it  considered  valid. 

In  order  to  use  the  discriminant  function,  training  data  was  required,  and  this  consisted 
of  algorithm  output  (Version  1.5)  for  the  three  sites,  with  each  rotation  region  declaration 
being  marked  as  valid  or  false  by  a  comparison  with  truth.  This  formed  the  basis  for  the 
model,  and  the  results  of  calculating  the  feature  means  and  standard  deviations  are  shown 
in  Table  10.  The  features  chosen  were  the  region’s  velocity  differential,  the  number  of 
segments  in  the  region,  its  area,  maximum  shear  and  average  shear.  The  column  labelled 
P  is  a  measure  of  how  significant  the  feature  is  in  its  ability  to  distinguish  the  class  of 
a  detection. 

8.1.1  Results 

From  the  entries  in  the  P  column  it  can  be  seen  that  the  number  of  segments  is  the 
most  important  feature  in  determining  whether  an  detection  is  valid,  and  in  agreement  with 
the  parameter  tuning  research.  This  is  borne  out  by  the  fact  that  the  more  successful 
versions  of  the  algorithm  were  found  to  be  the  ones  that  declared  more  segments  on  the 
whole.  Figure  20  shows  the  distribution  of  valid  regions  by  the  number  of  segments  in 
them.  Compare  this  to  Figure  21,  which  shows  the  same  for  false  detections,  and  the 
discriminating  ability  of  this  feature  will  become  further  apparent. 

If  this  approach  is  to  be  used  as  an  improvement  to  the  algorithm,  it  will  be  best  to 
train  it  on  the  data  from  the  geographical  area  where  the  system  will  be  running  in  order 
to  obtain  the  most  appropriate  set  of  parameters.  To  obtain  an  upper  bound  on  the  perform¬ 
ance  improvements  that  can  be  expected,  it  was  possible  to  get  classification  information 
for  the  case  data  using  the  case  data  itself  as  a  training  model.  From  the  resulting  statistics 
given  in  Table  11,  it  can  be  inferred,  for  example,  that  for  Denver  the  number  of  false 
detections  could  decrease  by  21  percent,  at  an  expense  of  only  1  percent  of  its  POD. 
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Table  10. 

Discriminant  Analysis  Calculations 


False  detections  Valid  detections 

Kansas  City 


Feature 

mean 

stdev 

count* 

• 

• 

• 

mean 

stdev 

count 

T2 

vel  diff 

12.97 

3.33 

1092 

ft 

ft 

ft 

12.94 

2.28 

439 

num  segs 

18.26 

12.66 

1092 

ft 

ft 

28.09 

14.26 

439 

area 

4.22 

3.24 

1092 

ft 

6.43 

3.50 

439 

max  shear 

7.15 

3.02 

1092 

ft 

7.13 

2.58 

439 

avg  shear 

6.59 

2.22 

1092 

ft 

ft 

6.72 

1.68 

439 

Denver 

ft 

Feature 

mean 

stdev 

count 

ft 

ft 

ft 

mean 

stdev 

count 

T2 

vel  diff 

13,24 

3.91 

927 

ft 

ft 

ft 

14.06 

2.62 

108 

2.130 

num  segs 

16.22 

12.38 

927 

ft 

ft 

22.00 

12.80 

108 

4.575 

area 

4.64 

3.28 

927 

ft 

ft 

5.81 

3.85 

108 

3.419 

max  shear 

5.96 

2.97 

927 

ft 

ft 

6.61 

2.40 

108 

2.164 

avg  shear 

5.84 

2.29 

927 

« 

• 

6.36 

1.80 

108 

2.256 

Hunt8>4lle 

• 

Feature 

mean 

stdev 

count 

ft 

ft 

ft 

ft 

mean 

stdev 

count 

T2 

vel  diff 

13.38 

3.30 

488 

ft 

ft 

14.31 

3.38 

49 

1.868 

num  segs 

9.43 

8.28 

ABB 

ft 

ft 

24.92 

12.46 

49 

11.800 

area 

3.10 

2.39 

488 

ft 

ft 

5.96 

3.23 

49 

7.682 

max  shear 

7,46 

2.99 

488 

ft 

ft 

8.41 

3.62 

49 

2.070 

avg  shear 

6.86 

2.29 

488 

ft 

ft 

7.37 

1.97 

49 

1.497 

*  The  count  Is  the  number  of  objects  In  each  class  used  to  calculate  Its  mean. 
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frequency 


number  of  segments 


Figure  20.  Number  of  Segments  Distribution:  VALID  Detections 

The  figure  shows  the  distribution  of  the  number  of  segments  for  valid  rotation  region  declarations. 


57 


0  3  6  9  12  15  18  21  24  27  30  33  36  39  42  45  48  51  54  57  60  63  66  69  72  75  78 


number  of  segments 


Figure  21.  Number  of  Segments  Distribution:  FALSE  detections 

The  figure  shows  the  distribution  of  the  number  of  segments  for  false  rotation  region  daclaratlons. 
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Table  11. 

Results  of  Discriminant  Anaiysis 


Site 

valid  regions 
kept 

valid  regions 
discarded 

false  regions 
kept 

false  regions 
discarded 

Kansas  City 

417 

22 

888 

204 

Denver 

107 

1 

732 

195 

Huntsville 

46 

3 

175 

313 

8.1.2  Conclusions 

A  few  points  can  be  made  here  with  regard  to  incorporating  a  discriminant  capability 
into  the  rotation  algorithm.  One  is  the  importance  of  acquiring  training  data  for  the  site 
where  the  system  will  be  run,  as  was  seen  with  the  parameter  tuning.  Results  from  a  site 
that  is  meteorologically  similar  could  also  be  used.  It  should  also  be  noted  that  although 
the  implementation  of  the  discriminant  function  may  be  useful,  it  may  also  be  computation¬ 
ally  expensive  for  use  in  a  real-time  system.  The  calculation  to  be  used  according  to  this 
analysis  involves  the  multiplication  of  a  5X5  matrix  with  two  vectors,  for  a  total  of  at 
least  122  operations  to  be  performed  for  the  classification  of  each  detection.  Whether  this 
can  be  performed  quickly  enough  in  real  time  is  a  function  of  the  computer  system  used. 


It  is  more  significant  that  the  final  results  agree  with  the  parameter  tuning  approach 
in  that  the  number  of  segments  in  a  rotation  region  figures  strongly  in  its  chances  of  being 
a  successful  detection.  Another  ramification  of  this  result,  however,  is  that  the  number 
of  segments  is  also  a  feature  that  can  vary  greatly  with  the  geographical  source  of  the  data, 
and  is  the  most  discriminating  feature  found  in  the  analysis.  Since  this  technique  requires 
training  data  before  the  algorithm  can  be  made  operational,  it  is  necessary  to  generate  truth 
for  every  proposed  site  where  the  algorithm  will  be  used.  If  this  practical  consideration 
cannot  be  accommodated,  then  adding  a  discriminant  capability  to  the  rotation  algorithm 
would  not  be  worthwhile. 
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9.  SUMMARY  AND  CONCLUSIONS 


Tests  show  that  parameter  tuning  can  dramatically  alter  the  performance  of  the  rotation 
algorithm.  Although  the  algorithm  thresholds  chosen  to  be  varied  made  theoretical  and 
meteorological  sense,  sometimes  they  did  not  seem  to  have  much  impact  unless  testing 
results  were  separated  by  site.  It  is  important,  therefore,  that  parameter  testing  be  done 
on  a  homogeneous  set  of  test  data,  that  is,  all  from  the  same  meteorologically  similar  region, 
in  order  to  obtain  appropriate  parameter  settings.  This  is  evident  because  various  algorithm 
sensitivities,  such  as  the  presence  of  strong  environmental  winds  or  low  signal-to-noise 
ratios,  are  indigenous  to  particular  geographical  areas.  Although  this  data  segregation  is 
critical  to  achieving  maximum  algorithm  performance,  for  experimental  reasons  the  data 
used  for  this  study  was  a  mixture  of  three  sites.  Nevertheless,  the  results  were  a  detection 
probability  of  84  percent  and  a  false  detection  rate  of  37  percent  for  the  higher  reflectivity 
regions.  Tests  on  a  smaller  statistical  scale  point  strongly  toward  a  much  higher  POD  (up 
to  100  percent)  and  lower  PFD  (as  low  as  17  percent)  when  algorithm  parameter  adjust¬ 
ments  are  made  with  regard  to  the  particularities  of  a  radar  site.  Furthermore,  the  logic 
and  parameter  changes  made  to  the  algorithm  resulted  in  a  lower  detections-to-truth  ratio, 
signifying  improvement  to  the  discretion  of  the  algorithm’s  decision-making  process. 

The  major  focus  of  the  algorithm  improvement  process  was  to  decrease  its  false  detec¬ 
tion  rate,  and  this  was  successful.  Without  benefit  of  reflectivity  or  stortn  cell  information, 
though  the  algorithm  can  always  detect  azimuthal  shear;  it  may  not  necessarily  be  a  rotation 
region  associated  with  a  microburst.  Thus,  the  PFD  can  be  lowered  a  limited  amount. 

The  results  of  a  preliminary  discriminant  analysis  show  that  if  the  discriminant  function 
were  to  be  included  in  the  algorithm’s  implementation,  27  percent  of  the  number  of  false 
detections  would  be  eliminated  while  the  number  of  valid  regions  lost  would  represent  only 
seven  percent  of  the  POD.  The  analysis  showed  that  the  number  of  segments  in  a  region 
is  its  most  distinguishing  characteristic,  although  other  features,  such  as  shear,  area  and 
velocity  differential  should  be  included  in  the  covariance  matrix  calculation. 

The  main  conclusion  is  that  segments  should  be  allowed  to  be  shorter  and  more  numer¬ 
ous,  while  at  the  same  time  the  azimuthal  association  criteria  should  be  loosened.  In  this 
way,  spurious  segments  will  be  grouped  together  into  regions  that  ultimately  will  be  invali¬ 
dated  because  of  insufficient  area,  while  legitimate  rotation  will  be  detectable,  even  though 
the  data  may  be  noisy. 

9.1  RECOMMENDATIONS 

The  shear  validation  test  with  its  shear__drop  threshold  was  successful  in  helping  to 
reduce  the  PFD  overall,  and  it  should  be  included  in  the  next  release  of  the  rotation  algo¬ 
rithm.  Furthermore,  parameter  tuning  should  be  done  after  testing  has  been  done  on  data 
from  that  meteorological  region  only.  Data  from  different  sites  should  not  be  combined 
when  running  tests  for  threshold  adjustments.  The  final  parameter  set  determined  by  this 
study  is  superior  to  the  currently  operational  one,  and  it  represents  a  good  starting  point. 
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The  technique  used  for  parameter  optimization  was  basically  heuristic,  and  it  worked 
well  for  the  set  of  parameters  used  because  intimate  knowledge  of  the  algorithm’s  behavior 
could  be  employed  as  the  the  parameters  were  adjusted.  An  alternative  technique  would 
be  to  organize  the  various  measures  of  algorithm  performance  into  a  function  and  then 
maximize  this  function  in  n-space  (where  n  is  the  number  of  relevant  parameters  to  be 
adjusted)  using  a  gradient  method.  The  disadvantage  of  this  method  is  that  it  may  be  im¬ 
practical,  requiring  too  many  iterations  of  running  the  algorithm  and  grading  its  perform¬ 
ance  on  large  data  set. 

Discriminant  analysis  can  be  directly  useful  for  significant  false  detection  rate  reduction 
if  incorporated  into  the  algorithm.  It  also  can  be  indirectly  useful  for  close  inspection  of 
characteristics  of  algorithm  output,  which  can  provide  information  on  further  improvements 
to  the  POD. 

Dual  Doppler  analysis  would  also  be  helpful  in  the  characterization  of  a  valid  rotation 
region.  The  dual  Doppler  windfield  could  be  compared  to  the  single  Doppler  image,  which 
could  provide  some  insight  on  how  to  interpret  the  rotation  image,  resulting  in  further  reduc¬ 
tion  of  the  algorithm’s  PFD. 

If  the  effects  of  strong  environmental  winds  could  be  filtered  out  of  the  radar  image, 
the  rotation  algorithm  could  then  be  made  to  search  for  velocity  runs  of  negative  to  positive 
(and  positive  to  negative),  as  opposed  to  just  increasing  or  decreasing,  which  then  bears 
a  closer  resemblance  to  the  canonical  radar  signature  of  rotation  as  depicted  in  Figure  4b. 
Though  other  causes  of  false  detections  would  still  exist,  the  large  number  due  strictly  to 
an  environmental  wind  effect  could  be  reduced. 

More  research  is  needed  to  arrive  at  an  optimum  value  for  the  shear_firop  threshold. 
It  may  be  possible  to  use  simulated  data  to  quantify  the  second  derivative  changes  in  the 
sine-like  function  of  the  model  signature  to  get  the  proper  value.  However,  a  good  statistical 
sampling  of  data  for  the  site  in  question  may  be  a  better  alternative. 

If  the  data  for  entire  scan  can  be  stored  and  retrieved  efficiently  in  real-time,  then 
the  pattern  matching  process  of  the  rotation  algorithm  can  be  extended  to  two  dimensions. 
Since  far  more  research  in  general  has  been  done  on  two-dimensional  images,  this  opens 
up  many  possibilities  for  using  more  well-known  and  stable  image  processing  techniques, 
such  as  a  two-dimensional  matched  filter.  Furthermore,  one  would  then  be  able  to  take 
advantage  of  available  reflectivity  information,  assuming  it  could  also  be  processed  for  an 
entire  scan. 

Investigation  into  the  use  of  the  spectrum  width  product  of  the  radar  may  be  fruitful. 
This  has  a  good  possibility  of  success  because  of  the  high  variance  of  velocities  in  most 
rotation  regions  in  the  vicinity  of  downdrafts. 
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GLOSSARY 


TDWR 

PRF 

isodop 

bounding  boxes 

shear  segments 
AGL 

truth  (ground  truth) 

PFD 

POD 

DTR 

ASP 

RR 

event 

SNR 

wet 

dry 


Terminal  Doppler  Weather  Radar 
Pulse  Repetition  Frequency 

A  line  connecting  points  of  the  same  Doppler  velocity. 

Axis  aligned  polygon  formed  by  connecting  the  four 
coordinates  (x  min,  y  min,  x  max,  y  max)  which  give  the 
x-y  extrema  of  a  region. 

Lines  connecting  points  of  increasing  or  decreasing  velocity. 
Above  Ground  Level 

Regions  of  interest  as  determined  by  a  human  expert. 

Probability  of  False  Detection 

Probability  of  Detection 

Detection-to-Truth  Ratio 

Automatic  Scoring  Program 

Event  Recognition  Rate 

A  group  of  truth  observations  occurring  over  time  within  the 
life  span  of  a  microburst. 

Signal-to-Noise  Ratio 

High  reflectivity  (above  45  dbZ) 

Low  reflectivity  (15  dbZ) 
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APPENDIX  A: 


SYNOPTIC  SUMMARIES 

The  following  tables  list  the  characteristics  of  all  the  microburst  events  on  each  case 
date  in  this  study.  Note  that  in  all  the  following  tables,  the  reflectivity  is  the  value  of  the 
core  reflectivity  at  the  surface. 


July  25, 1986,  Huntsville,  AL  —  Seven  high-reflectivity  microbursts  and  one  gust  front 
occurred  on  this  day,  which  was  hot  with  characteristic  air-mass  thundershowers. 


Microburst  Characteristics 


Time 

(COT) 

Location 

Rng/Az 

Intensity 

m/s 

Reflectivity 

dbZ 

1354 

25/184 

15 

55 

1519 

21/251 

12 

55 

1519 

27/253 

12 

n/a 

2004 

20/215 

21 

55 

2040 

20/142 

21 

55 

2148 

4/124 

27 

60 

2206 

23/075 

21 

55 

September  2,  1987  Denver,  CO  —  There  were  1 1  microbursts  and  four  gust  fronts 
on  this  day,  which  was  warm  with  thundershowers.  By  noon,  a  line  of  thundershowers 
extend  along  the  front  range  [of  the  Rocky  Mountains] .  Several  microburst  scale  outflows 
were  detected  over  the  mountains.  At  1250  MDT  a  gust  front  was  scanned  west  of  Stapleton 
International  Airport  in  Denver.  A  weak  divergent  line  had  developed  by  1300  MDT  (see 
below). 
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Microburst  Characteristics 


Time 

(CDT) 

Location 

Rng/Az 

Intensity 

m/s 

Reflectivity 

dbZ 

1123 

61/245 

16 

40 

1233 

42/261 

18 

40 

1255 

24/266 

12 

15 

1304 

30/249 

12 

10 

1616 

26/305 

24 

20 

1630 

31/016 

15 

60 

1649 

23/309 

24 

20 

1659 

6/322 

18 

15 

1703 

9/356 

21 

20 

1745 

27/057 

24 

40 

1751 

33/042 

24 

25 

July  1,  1989,  Kansas  City,  MO  —  On  this  day,  there  were  eight  microbursts  and  one 
gust  front.  The  combination  of  a  vorticity  lobe  and  low-level  moisture  in  the  Kansas  City 
area  touched  off  thundershowers  during  the  mid-afternoon  hours.  By  2230,  several  weak 
to  moderate  microbursts  were  detected  to  the  northeast  and  southeast  of  the  radar.  A  weak 
microburst  at  2215  set  off  a  wind  shear  alert  at  the  airport. 


68 


Microburst  Characteristics 


Time 

(CDT) 

Location 

Rng/Az 

Intensity 

m/s 

Reflectivity 

dbZ 

1959 

26/037 

19 

54 

2009 

26/141 

10 

40 

2050 

6/092 

11 

45 

2100 

22/076 

16 

45 

2117 

30/076 

10 

n/a 

2145 

16/067 

10 

45 

2215 

13/040 

11 

45 

2224 

20/190 

10 

n/a 

July  30, 1989,  Kansas  City,  MO  —  The  day  was  warm  with  thundershowers,  and  there 
were  19  microbursts  and  three  gust  fronts.  An  abundant  supply  of  moisture  and  an  outflow 
boundary  from  central  Missouri  combined  to  produce  moderate  to  severe  thunderstorms 
in  the  Kansas  City  area  during  the  early  morning  hours.  Several  moderate  to  strong  micro¬ 
bursts  were  observed  to  the  east  and  northeast  during  this  time.  At  0930  UT,  the  “Clayco- 
mo”  microburst,  with  a  total  shear  of  40  m/s,  was  observed  at  29/093.  This  microburst 
downed  several  large  trees  and  damaged  several  trailer  homes. 
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Microburst  Characteristics 


Time 

(CDT) 

Location 

Rng/Az 

Intensity 

m/s 

Reflectivity 

dbZ 

0710 

15/110 

30 

40 

0723 

17/103 

21 

35 

0727 

26/084 

24 

40 

0734 

22/107 

12 

40 

0736 

32/072 

15 

42 

0742 

20/110 

15 

40 

0751 

29/100 

11 

40 

0757 

24/021 

16 

35 

0818 

23/051 

11 

35 

0851 

22/075 

10 

35 

0859 

20/099 

12 

43 

0908 

24/073 

12 

30 

0914 

23/098 

14 

42 

0927 

25/103 

10 

40 

0928 

28/092 

40 

45 

0930 

23/084 

15 

30 

0941 

23/088 

12 

40 

0947 

30/102 

15 

40 

0946 

23/087 

10 

40 
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